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Abstract 

Undersea technology is on the verge of equipping remotely operated vehicle (ROV) pilots 
with a three-dimensional (3-D), real-time display incorporating data from a wide variety of 
sensors including sonar (sound navigation and ranging), cameras, and lasers. Effective 
collection, computation, and presentation of this data to the pilot in a single display presents 
hardware, software and human factors problems. This thesis focuses on human factors 
issues associated with the display of information which could enhance the pilot's efficiency 
of performance. Background information on human factors engineering, 3-D computer 
graphics displays, and application of the 3-D perspective display precede the details of the 
experiment. 

Five specific display enhancements tested include altering the displayed field of view, 
providing a screen grid, displaying the current range to the target of interest, using a 
vertical color scheme, and controlling the display update rate. Seven tests measure the 
effects of these display enhancements on the simulated piloting of an ROV. The effects of 
the ROV simulation and operator learning curves are removed to compare performance 
changes due to the various enhancements directly. Operator comments during and after 
testing as well as test monitor/author observations provide insight into the experiment. Test 
result implications for system design trade-offs are discussed in detail. Recommendations 
for future research and the proposed construction of a fully equipped ROV simulator 
complete the work. 
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Chapter 1 



Introduction 



1.1 Background 

Sonar (Sound Navigation and Ranging) is a technique for underwater range finding 
that has existed for more than fifty years. The author's thirteen years of experience with 
military and commercial sonars indicate that the techniques employed to present sonar 
information to the user significantly lag the technology used in areas such as hardware 
design, beam forming, and data processing, and unnecessarily limit the bandwidth of 
information transfer. The wide variety of display designs and information formats proves 
confusing to all but the most adept and experienced users. Additionally, the typical 
restriction of the presentation to two coding dimensions, coincident with the physical 
dimensions of the cathode ray tube (CRT), severely restricts the ability to effectively 
convey large amounts of data. 

The Woods Hole Oceanographic Institution's (WHOI) Deep Submergence Labora- 
tory (DSL) operates several seagoing craft including remotely operated vehicles (ROVs), 
whose pilot "flies" the deep-diving ROV from an operations center located on a mother ship 
on the surface above. Within video camera range the pilot’s display resembles the view a 
driver sees out the front windshield of an automobile, with some loss of depth perception. 
This video display format is "natural" for the pilot and provides a large information transfer 
bandwidth. Suspended sediment near the bottom frequently limits the already restricted 
visual range of camera equipment to a couple of meters. Beyond visual range the pilot 
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must rely on the customary two-dimensional (2-D) active (transmit and receive) sonar 
presentations. 

Technological advances currently provide the capability to combine undersea data 
from a number of sensors including active sonars, video tape and electronic still cameras, 
manipulators, and laser range finders. These data can be assimilated and processed in real 
time to provide large quantities of high quality information. The data generated can be used 
to provide the ROV pilot with a real-time, three-dimensional (3-D), perspective display 
similar to that of the video camera when beyond visual range. This more natural view 
would ease the pilots workload and improve the efficiency of work conducted beyond 
visual camera range. Such a perspective-view display is expected to be dominated in the 
foreground by high-resolution video data, gradually transitioning to dominance by other 
sensors as the limits of video range are encountered. The envisioned 3-D display must be 
carefully designed to best improve the ROV pilot's efficiency. 

1.2 Research Objectives 

A variety of display enhancements have been used in military and commercial 
sonars. This thesis looks at the individual effects of six specific display enhancements: 
widening the field of view (FOV) to provide more information to the pilot, 2-D and 3-D 
gridding for improved angular and depth perception, vertical color schemes for better 
altitude versus depth perception, direct range readout to shorten the search and location 
cycle, and shortening the display update rate. Seven experiments to isolate and quantify the 
individual effects of these enhancements are described in detail. The analysis allows direct 
comparison of the effectiveness of the six enhancements. 
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1.3 Terminology 



Terminology is critical to comprehension. Several important definitions include: 

• altitude - the distance above the floor of the ocean. 

• data - "raw" numbers from which information can be extracted. 

' • data point - one of a set of 25 x, y, and z coordinates chosen for a target course. 

• depth - the perceived distance into a video or computer display. This is a spatial 

perception of range. 

• display - the computer screen dedicated to graphics information. 

• driving - the act of operating the simulated ROV for testing. 

• heading - the compass direction in which the vehicle is pointed. 

• monitor - the computer screen dedicated to printed information, including update 

indication, "target hit" and "mission complete" indication, target range 
indication (test 6), and vehicle world-coordinate position, heading, and 
velocities. This term is also used to describe the actions of the author as 
the testing supervisor. 

• operator - a test subject who drove the simulated ROV. 

• pilot - a qualified flyer of operational ROVs. 

• piloting - the act of flying a operational ROV. 

• target - the numbered, cubic wireframe, data point visualizing tool. 

• target course - one of 19 (lettered A-R, X) specific sequences of data points and 

associated targets. 

• update - the complete refreshing of a computer screen. 

• vertical - the direction opposite to that in which gravity acts. A vertical color 

scheme encodes altitude information using a color scale or look-up table. 
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1.4 Overview 



Chapter 2 provides background information on human factors engineering and its 
relationship to the thesis work. 

Chapter 3 contains information on three-dimensional computer graphics displays in 
general, and the display and equipment used in the thesis experiments. 

Chapter 4 discusses current and projected applications of three-dimensional 
perspective displays in underwater visualization. 

Chapter 5 describes in detail the conduct of seven tests including restricting the field 
of view, grids to provide depth and angular cues, color schemes to provide altitude cues 
versus depth of field, direct range information, and slowing of the graphic display update 
rate. 

Chapter 6 presents the results of the seven tests including operator comments and 
author/test monitor observations. 

Chapter 7 summarizes the results of the experiments, discusses conclusions, and 
offers recommendations for future work. 
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Chapter 2 



Human Factors Engineering 



2.1 Objectives and Doctrines 

Human factors engineering is the study of people and how equipment designs affect 
them. The two major objectives of human factors engineering are to improve efficiency 
and effectiveness and to promote key elements such as safety, satisfaction, and stress 
reduction. Six doctrines that form a set of human factors commandments are [S-l]: 

• Design with the user in mind. 

• Recognize differences in capabilities and limitations of people and their design 

implications. 

• Design influences human behavior. 

• Design process includes empirical data and evaluation. 

• Reliance on scientific methods and objective data. 

• Commitment to system orientation. 



2.2 Methodology Issues 

Experimental research methods test the effects of variables on behavior. Of prime 
interest in this work is the behavior "efficiency of operator performance". The method 
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selected needed to accurately measure this efficiency. Eight important issues associated 
with the choice of methodology are [M-l]: 

• Effectiveness. 

• Ease of use. 

• Cost. 

• Flexibility. 

• Range. 

• Validity. 

• Reliability. 

• Objectivity. 

No approach was best for all the issues above. Among them effectiveness, 
validity, reliability, and objectivity were deemed most critical for this thesis. Validity, in 
particular, was enhanced by input from experienced ROV pilots and operational support 
group personnel. These inputs served to more realistically match the simulation to 
operating conditions. 



2.3 Research Criteria 

Research criteria can be divided, with inevitable overlap, into the areas of system- 
descriptive criteria, task-performance criteria, and human criteria [M-l]. The criteria 
chosen for analysis in this work was task performance, specifically performance time. 
Human criteria cannot be ignored, however, and provide insight in the form of subjective 
comments by the operators, both during and after each test, and observations by the test 
monitor/author. 
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2.4 Visual-Coding 



Color is a prime visual-coding technique. It has been frequently used in 3-D 
underwater visualization to provide depth/relief cueing in the form of a continuous color 
scale versus depth. In this work two discrete color sets were chosen for testing. Both the 
6- and 21-color sets were derived from the red, green, and blue (RGB) color model. 

Figure 2.1 shows the RGB cube [F-l]. The 6-color set was selected by following a path 
from comer to comer in the following order: red, yellow, green, cyan, blue, and magenta. 
The 21 -color set followed the same path, selecting colors at each 0.25-unit path-length 
increment. 

The designation of specific color names was complicated by the existence of a 
variety of descriptive systems. Among the more notable were RGB, HSB, Munsell, 
ISCC-NBS, YIQ, CIE, and UCL [B-l, F-l, K-l], Target color names were derived 
starting with the basic six colors of the RGB color set (identical to the six basic hues of the 
HSB color set): red, yellow, green, cyan, blue, and magenta [F-l], The further 
subdivisions were named in accordance with the spirit of the Color Naming System 
(CNS), which was developed to simplify color descriptions [B-l]. The 21 color names 
derived were used for the 1-, 6-, and 21-color sets. Table 2.1 lists the colors by name and 
includes RGB and HSB descriptions. Conversions between most color sets were available 
and easily done with a computer program [F-l]. 



2.5 Control and Tracking Performance 

Controls transmit discrete or continuous signals to a system to produce a desired 
response. Important factors in control design include ease of identification, size, control- 
response ratio, resistance, lag, deadspace, backlash, location, and compatibility [S-l]. 
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Figure 2. 1 



Blue = (0, 0, 1) Cyan = (0, 1, 1) 




Table 2. 1 Color set RGB to HSB conversion 



Color 


Red 


Green 


Blue 


Hue 


Saturation 


Brightness 


Red 


1 


0 


0 


0 


1 


1 


Reddish Orange 


1 


0.25 


0 


15 


1 


1 


Orange 


1 


0.50 


0 


30 


1 


1 


Yellowish Orange 


, 1 


0.75 


0 


45 


1 


1 


Yellow 


1 


1 


0 


60 


1 


1 


Yellowish Yellow-Green 


0.75 


1 


0 


75 


1 


1 


Yellow-Green 


0.50 


1 


0 


90 


1 


1 


Greenish Yellow-Green 


0.25 


1 


0 


105 


1 


1 


Green 


0 


1 


0 


120 


1 


1 


Greenish Aquamarine 


0 


1 


0.25 


135 


1 


1 


Aquamarine 


0 


1 


0.50 


150 


1 


1 


Cyanish Aquamarine 


0 


1 


0.75 


165 


1 


1 


Cyan 


0 


1 


1 


180 


1 


1 


Cyanish Turquoise 


0 


0.75 


1 


195 


1 


1 


Turquoise 


0 


0.50 


1 


210 


1 


1 


Blueish Turquoise 


0 


0.25 


1 


225 


1 


1 


Blue 


0 


0 


1 


240 


1 


1 


Blueish Purple 


0.25 


0 


1 


255 


1 


1 


Purple 


0.50 


0 


1 


270 


1 


1 


Magentaish Purple 


0.75 


0 


1 


285 


1 


1 


Magenta 


1 


0 


1 


300 


1 


1 
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Deadspace refers to a null established about the control's normal position in which control 
device motion results in no signal output. Backlash is deadspace at all control positions. 
The control mechanism may have no resistance (free-positioning or pure-displacement 
control), no displacement (stiff stick or pure-force control), or lie in the middle (which we 
term a mixture control). 

The control order of a system refers to the relationship between the control 
mechanism and the system output. Orders of control include [S-l]: 

• position (zero-order) control - where the signal controls the output directly. 

• velocity (first-order) control - where the signal controls the rate of change of 

position. 

• acceleration (second-order) control - where the signal controls the rate of change 

of velocity. 

• higher-order control - where the signal controls the rate of change of acceleration 

or other higher order term. 

The system in these tests used second-order control of thrust, equivalent to acceleration, in 
each of the four degrees of freedom (see Section 5.3). 

The problem presented to the operator was a compensatory tracking problem with 
time lag [S-l]. The time lag was composed of three parts: 

• response lag - the time required for the operator to choose and execute a 

response. 

• control system lag - the time required for the system to respond to commands 

including vehicle motion simulation. 

• display system lag - the time required for graphics calculations and display 

update. 

In each case the time lag increased tracking error. 
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2.6 Test and Evaluation 



Independent variables (IV s) are those parameters controlled by the experimenter. 
Dependent variables (DVs) are measured parameters affected by the IVs. They are the 
same as the research criteria discussed in Section 2.3. 

Evaluation of test results must consider test subjects, criteria, and experimental 
procedures [S-l]. Subjects should be representative of projected users. Dependent 
variables (criteria) must be related to system operational use. Experimental procedures and 
controls should give repeatable results. 
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Chapter 3 



Three-Dimensional Computer Graphics Displays 



3.1 True 3-D Displays 

Technology stands at the threshold of true three-dimensional displays. Available 
implementations display depth instead of simply providing depth cueing. Two such 
systems are SpaceGraph ™ by Bolt, Beranek, and Newman (BBN) Laboratories 
Incorporated of Cambridge, Massachusetts and OmniView™ by Texas Instruments (TI) of 
Dallas, Texas. 

The SpaceGraph™ system is based on reflecting a point plotted (not raster scanned) 
image in a vibrating mirror. The technique is termed the varifocal mirror. As the reflecting 
surface vibrates it repeatedly changes shape from concave to convex, causing a large 
change in focal length. When the image update rate is synchronized with the mirror 
vibration, a true 3-D image is created in space. For many data-set analyses the superior 
pattern -recognition capability of the human brain is enhanced by this type of display. This 
system is limited in viewing angle, display update rate, and to simple monochrome images. 
[F-l, T-l, S-2] 

The OmniView ™ system is based on laser scanning of a paraboloidal, rapidly 
rotating surface. During rotation, the scanning laser beam's reflections form the 3-D 
image. Advantages include 360° horizontal viewing and total wrap-around parallax. This 
system has no hidden surface removal and cannot display solid surfaces. [E-l] 
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Holography is another promising technology for true 3-D viewing. This work is 
based on the splitting of a laser beam into two parts, illumination of objects by one of the 
split beams, and their subsequent recombination using the second beam as a reference. 
Limitations typically include small size and restricted viewing angle, lack of parallax, and 
high cost. [E-l, F-l] 

Head-mounted displays have offered tremendous potential since their birth in the 
late 1960s [S-3, S-4]. The helmeted user enjoys freedom of movement in a virtual 3-D 
world projected in real time for separate viewing by the each eye. Technical problems have 
severely restricted the development of this technology since its inception [F-l]. 

An inherent limitation of true three-dimensional viewing is the tremendous quantity 
of data and associated data processing, particularly critical for real-time display. Expenses 
for computation rise dramatically as realistic, complex 3-D imagery is approached and 
parallel-processing supercomputers become a necessity. Significant effort and funding are 
being funnelled into this cutting-edge technology, but it is not likely to be available to the 
field scientist before the late 1990s. 



3.2 Simulated 3-D Displays 

In the absence of true 3-D displays several methods are employed to simulate three- 
dimensional viewing using two-dimensional CRTs. These methods are designed to 
counter the information loss inherent in projecting 3-D information on a 2-D surface such 
as a computer screen. Techniques used include those for line drawings and shaded images, 
as well as dynamics, stereopsis, and head-motion parallax. 

Line drawing techniques include the use of various projections, depth cueing with 
intensity, depth clipping, and hidden-line removal [F-l]. Among the projections, the 
perspective offers two significant depth cues: decreasing size of objects with depth and the 
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convergence of parallel lines to their vanishing points. Depth cueing with intensity draws 
the more distant objects at a lower intensity. The result is a strong depth cue similar to that 
observed when viewing a scene in a light fog. Depth clipping limits the view by preventing 
the drawing of lines not within the designated limits of the front or back clipping planes. 
This effect is used to limit the depth of field. Hidden line removal calculates the lines that 
should not be visible based on the view and does not draw them. 

Shaded-image techniques include hidden-surface removal, control of illumination 
and shading, various shading interpolation methods, and the use of texture, shadows, 
transparency, and reflection. 

Dynamics refers to the added information available when a sequence of views is 
shown instead of a static view. The primary depth cue is provided by the kinetic depth 
effect, the lower velocity of nearby objects versus those in the background. 

Stereopsis is a depth cue taking advantage of the binocular disparity between human 
eyes [F-l]. The ability of the human brain to fuse two slightly differing views of a scene 
and extract depth information is a powerful depth cue. 

Head-motion parallax uses the changing view as occurs in real life when the head is 
moved side to side. 

3.3 The Marquest System 

Several important components of the test equipment hardware and software were 
provided by the Marquest Group, Inc. of Bourne, Massachusetts. They include the 
joystick module and associated software, an Nth Engine/550™ Display Controller, and 
HOOPS™ graphics routines and HYDRA™ rendering software for the Nth Engine/550™. 
They are components of a commercial ROV control system. 

The joystick module is a standardized control device for ROVs constructed by the 
Marquest Group, Inc. using off-the-shelf components. Included are a trackball, bias and 
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gain potentiometers, nine push-button switches, and a 3-axis joystick. Only the joystick 
was used in the testing. Measurement Systems, Inc. of Norwalk, Connecticut, built the 
industry standard, heavy duty, 3-axis (x, y, 0) joystick with mixture control and deadspace 
on each of the three axes. A thumb switch provided the fourth axis of freedom (z) with 
pure force control. Another joystick available from Measurement Systems, Inc., but not 
used here, incorporated the fourth axis of freedom through lifting/depressing the joystick 
vertically with mixture control. Joystick use is discussed in Chapters 6 and 7. 

The Nth Engine/550™ Display Controller is a 3-D graphics processor/controller 
built by Nth Graphics of Austin, Texas. The controller has 1280 x 1024 resolution and 
operates at 20 MIPS and 3 MFLOPS. The board set includes an NthTV™ converter 
module, which provides a standard commercial television format (NTSC) tap for direct, 
real-time recording of the video display on a video cassette recorder (VCR). 

HOOPS™ from Ithaca Software of Ithaca, New York, formed an interactive three- 
dimensional graphics library. This software controlled the Nth Engine/550™. 

HYDRA™ rendering software, another product of Nth Graphics, was used to 
convert targets and grids drawn in AutoCAD® Release 10 to the required format for the 
HOOPS graphics routines. A helpful feature is the ability to quickly conduct a visual check 
of targets and grids. 



3.4 The Display and the Monitor 

Planar geometric projections fall into two categories: perspective and parallel. 
Characteristics which distinguish a perspective projection from a parallel projection are 
convergence of parallel lines, size diminution, and nonuniform foreshortening [C-l]. The 
primary advantage of a perspective projection is the natural appearance of objects seen by 
the eye. It was also the standard video camera projection. For these reasons a three-point 
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perspective projection was chosen for the display [C-l]. The graphics were shown on a 
Sony model GDM-1950 19-inch color monitor with 1280 x 1024 pixel resolution. 

Of the previously discussed techniques to provide three dimensionality, depth 
cueing in particular, many were not available in the experimental system for three reasons: 
the tests used a perspective projection, the targets were drawn in wireframe mode, and 
limitations of the Nth Graphics boards and software running the display. Depth cueing 
was provided by target relative size (because they were all identical in dimension), the 
convergence of parallel lines (emphasized by the cubic wireframe targets), and the kinetic 
depth effect. 

An NEC VGA color monitor (with blue background) provided the following: 

• display update indicated by the word "Busy.." appearing in white letters in a red 

box. 

• most recent target hit indicated by the words "OBJECT #( target number) HIT" 

appearing in white letters in a red box. 

• test completion indicated by the words "MISSION COMPLETE" appearing in 

white letters in a red box. 

• range in meters from the vehicle (modelled as a point) to the center of the next 

target indicated by the symbol "R = (range)" appearing in white letters in a 
red box. This information was provided to the operator only during test 6. 

• world system coordinates (x, y, and z) and heading of the vehicle in white 

numerals. Titles (xpos, ypos, zpos, and hdg) were shown beside the 
coordinates/heading in white letters. 

• vehicle system velocities (x, y, z , and 0 rate) in white numerals. The titles 

(xvel, yvel, zvel, and rvel) shown in black letters above the velocities were 
individually accented with small red boxes if their magnitudes exceeded 
0.05 m/s or 0.05 rad/s. 

Figure 3.1 shows the monitor screen with range information during test 6 . 
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Figure 3.1 Monitor screen during test 6 
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Chapter 4 



Application of the 3-D Perspective Display 



4.1 Ocean Environmental Sensing 

Early sonar development was driven by military needs. Crude processing and 
display techniques limited these systems to deterministic methods with severely restricted 
dynamic range. Inherent in these limitations were a disregard for inaccuracies such as 
angular and range resolution, variations in the speed of sound and the sound ray paths, and 
navigation errors in position, attitude, depth and motion. The current state of technology 
overcomes many of these restrictions. Modem computers and software graphics packages 
allow real-time processing and display of more advanced models of the ocean environment. 
New commercial sonars provide more accurate information from a variety of sampling 
techniques including swath mapping, sidescan, and upward-looking sonars, in addition to 
the more traditional forward-looking and bathymetric sounding versions. Innovative 
electronic instruments provide high-accuracy navigational data. Recent trends have resulted 
in additional technologies being adapted for underwater work. Video and electronic still 
cameras, and scanning lasers are among these supplementary sensors. 
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4.2 Multisensor Modeling 



To effectively combine the various sensor modalities into a single model of the 
ocean environment, we must first accept the premise that no measurement is exact. Any 
information gathered will have an associated probability distribution, which is itself not 
known deterministically. The ocean model can be built incorporating these uncertainties as 
a stochastic model. This approach is especially effective when combining information from 
different sources with their differing ranges, resolutions, and data types. At any time, a 
deterministic model can be extracted from the current probabilistic one. Using this method, 
a more accurate deterministic model is obtained which accounts for the many and varied 
errors inherent in measurements. 

Initially, the ocean volume of interest is divided into a 3-D grid of cubic volume 
elements, or voxels. The size of the voxels is dependent upon the desired model resolution 
as well as system hardware and software limitations for real-time processing. Each voxel 
is assigned a "feature" vector incorporating information such as the time, the position of the 
voxel in the coordinate space, the measured characteristics of interest (features), and a 
probability or confidence value for each feature. As new information is collected, older 
data is not discarded. Feature vectors are incremented using a data merging technique 
called stochastic backprojection [S-5, S-6]. This data merging technique is an adaptation of 
the backprojection and summation method used to reconstruct images. The method is 
enhanced to include probabilistic uncertainty and allow stepwise solutions for real-time 
processing. Use of these techniques allows combination of the redundant feature data from 
all available sources, improving the model resolution and certainty. 
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4.3 The Deep Submergence Laboratory System 



The Deep Submergence Laboratory of the Woods Hole Oceanographic Institute 
employs a variety of sensors and platforms. A typical cruise involves flying equipment to a 
proximate port and loading out a chartered mother vessel. At the research site, a large area 
survey is conducted with a towed 120-kHz split-beam sonar system. This survey provides 
medium resolution imaging with a swath width of 200-300 meters at a tow speed of 6-10 
knots, and allows items of interest to be identified for subsequent revisiting with higher 
resolution equipment. Items of interest are reinspected using the tethered Jason ROV 
system. Jason is typically equipped with a SHARPS short-baseline navigation system and 
a sensor package including a 200-kHz sidescan sonar, a 300-kHz forward-scanning sonar, 
a video camera, an electronic still camera, and a laser rangefinder. Figure 4. 1 shows the 
range and resolution overlaps between sensors in the current suite. 

The Jason pilot has several sources of information available. If within visual range, 
the video camera and vehicle lighting system provide a natural, perspective view with 
extremely high bandwidth. This visual range is often restricted to a meter or less due to 
suspended sediment in the water, and is usually less than about 10 meters. Beyond visual 
range the forward-scanning sonar, the 200-kHz sidescan sonar, the laser rangefinder, and 
the SHARPS navigation system provide data to several displays with differing formats. 

The pilot navigates using a specially designed 2-D auxiliary display with limited bandwidth 
until within visual range [S-7]. 

This current method taxes the pilot, whose task is complicated by [S-7]: 

• low visibility, reducing video camera range and safe working speeds. 

• the narrow field of view of the video camera (approximately 40°). 

• the monocular vision of the video camera. 

• optical distortion of the video camera. 

• the difficulty of interpreting sonar presentation screens. 
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SURVEY SWATH WIDTH 




Figure 4. 1 DSL comparison of underwater remote-sensing systems 
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• the transient nature of the displayed information, requiring the pilot to mentally 

integrate the overall picture. 

• the inability to access the high-level, real-time environmental model. 

To enhance ROV piloting efficiency, a real-time, 3-D display is envisioned. The 
display will access the high-level environmental model and present the user with a view 
closely resembling that of the video camera. Because low or medium resolution data are 
available prior to the ROV entering the water, a rough perspective- view display can be 
constructed at any time for use by the pilot on the screen previously reserved for video 
camera information. Such a perspective-view display is expected to be dominated in the 
foreground by high-resolution video data, gradually transitioning to dominance by other 
sensors as the limits of video range are encountered. 



4.4 Targeted Human Factors 

Among the myriad of potential display enhancements to improve the proposed 3-D 
perspective display are: 

• altering the displayed field of view. 

• providing a screen grid. 

• displaying current range to the target of interest. 

• use of a vertical color scheme. 

• controlling the display update rate. 

Narrowing the displayed field of view reduces the necessary graphics calculations 
as the square of the reduction factor. This gain must be balanced against the loss of 
information to the user. 
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Providing a 2-D or 3-D screen grid can improve the user's angular perception, 
provide a cross-hair effect to ease direction finding and, with the 3-D version, provide 
important depth cues. 

Displaying current range to the target of interest can reduce the time to locate the 
target. This information is available in the environmental model. 

Vertical color schemes are frequently used to provide redundant relief cueing for 
underwater and land feature presentations such as charts and maps. They can be effective 
in giving altitude cues for objects in the background of perspective displays. 

Display update rate is limited by the data-collection cycle time of the sensors, data 
processing time, and graphics calculation/display time. Compromises can be made to 
shorten the update rate at the cost of information quality and quantity. 

This thesis attempts to quantify the effects of the task-related independent variables 
listed above. The dependent variables for the experiments are performance time, subjective 
comments by the operators, and observations by the test monitor/author. 
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Chapter 5 



Testing Display Effectiveness 

5.1 Equipment 

Three system hardware components were used directly by the operator: a computer 
monitor, a computer display, and a joystick. Other equipment transparent to the operator’s 
task included a 386 computer with keyboard and mouse, which ran the simulation and 
recorded the data, and various power supply and communications cabling. 



5.2 Data Point Courses 

Nineteen data courses (A-R, X) were used in testing. Courses consisted of 25 
sequentially numbered data points (Pj) in world coordinate system three space (x, y, z) 

with two constraints: 

(1) Consecutive points were separated by distance s = v [Ax]2+[Ay]2+[Az]2, where 

s was a constant equal to 5 meters. 

(2) A square window of given total angular width and height (3, with apex at point 

Pj and opening in the direction of a line from Pj.] through Pj, contained 
point Pj+]. 

Courses A, B, C and J, K, L used a 15°- square window, courses D, E, F and M, N, O 
used a 30° window, and courses G, H, I and P, Q, R used a 60° window. Course X, the 
training/familiarization course, used a 15° window. Additionally, nine of the courses were 
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mirror images of each other in the world coordinate system y/z plane. Mirror pairs were 
A/J, B/K, C/L, D/M, E/N, F/O, G/P, H/Q and I/R. Figure 5.1 and table 5.1 summarize 
the above information. 

Visualization of each data point was provided in the form of a cubic wireframe 
target one meter on a side with the invisible data point at its center. Target faces were 
parallel to the planes formed by the world coordinate system axes. Target color was blue 
except for tests 5 and 6 as discussed in section 5.7. Each target was numbered with white 
numerals (1-25) corresponding to the sequential number of the data point at its center. A 
target "hit" corresponded to closing the range to the target center/data point to 0.5 meters or 
less from any direction. Appendix C discusses the details of target design. Appendix D 
contains course layout data and photographs of the courses as viewed by the operator. 

Random generation of the data points for each course began with establishment of 
two coordinate systems, the world and the vehicle. Each system consisted of three 
mutually perpendicular axes x, y, and z, forming a right-hand coordinate system. The 
world system was fixed in space. In it, three angles were defined: £0, cp, and K for positive 
rotation about the x, y and z axes respectively. Positive rotation was defined such that 
positive CO (cp, K) rotated the positive y (z, x) axis into the positive z (x, y) axis through 90°. 
In the vehicle system, the operator was assumed to be looking down the positive y axis of 
the system at all times with the positive x axis to the right. Positive rotation was defined so 
positive 0 rotated the positive y axis into the positive x axis through 90°. In this way, 0 
had the same sense as true heading. Figures 5.2 and 5.3 depict these systems. For a 
specified window width (3, sets of two angles CO and K were randomly selected from the 
interval [-(3/2, (3/2], This process insured that the data points met constraint (2) above. 

Based on the above coordinate system definitions, rotations of the vehicle system 
were broken down into individual rotations of CO, cp, and K. Orthogonal orientation 
matrices were then defined for the decomposition as follows [M-2]: 
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The most general orientation matrix M for any combination of these three rotations 
was then [M-2]: 



M = M C 0 M { pM K 

COS (p COS K 
= -cos (p sin K 
sin (p 



cos (0 sin K + sin to sin cp cos k 

cos to cos K - sin to sin cp sin K 

-sin to cos tp 

sin to sin k - cos to sin tp cos k 

sin to cos K + cos to sin tp sin K 

cos to cos tp 



In our application pitch (to) and yaw ( k ) were allowed to vary while roll (tp) was 
not. The general case was then specialized by setting tp = 0 to obtain: 



L — M(p— o - 



cos K 
-sin K 
0 



cos to sin K 
cos to cos K 
-sin to 



sin co sin K 
sin to cos K 
cos to 



The matrix Lj defined the look direction from the point Pj to the next point Pj + i in 
the vehicle coordinate system. The matrix (Lj)* defined the transformation from the vehicle 
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coordinate system to that of the world. The origin point was taken to be Pq = [ 0,0,0 ]T 
The first data point was taken to be Pj = s[ 0,1,0 ]T, where s was the constant described in 
constraint (1). The value (0, 0) was assigned to the set (co, k) at x = 0 yielding Lq = l3 x 3> 
the three-dimensional identity matrix, and successive Ljs were generated using the above 

formula for matrix L and the j-th set of values (co, k). As a result of these specifications 
(Lq)* = 13x3 and successive (Lj)*'s were calculated from the equation (Lj + j)* = (Lj)*Lj. 
Successive data points were then generated using Pj + j = Pj + s(Lj)*Lj [0,1,0 ]T as the 

recursive algorithm. 



5.3 Modeling the Jason Vehicle 



The vehicle model was a dynamic simulation of the DSL ROV Jason. The model 
exhibited four degrees of freedom: x (left/right), y (forward/reverse), z (up/down), and 9 
(CW/CCW). Hydrodynamic drag was assumed to have quadratic and linear (damping) 
components. Balancing forces yielded the equation (shown for the x component) : 



(M + M a ) x + 



(M + M a )' 



l max 



x + Cd x I x I = T 



where, 

M was the vehicle's mass, M a was the vehicle's added mass, and (M + M a ) 

was termed the vehicle's extended mass. 

T max was a constant (2.0 s) defining the magnitude of the linear drag. 

Qd was the drag coefficient. 

T was the maximum thrust. 

Table 5.2 summarizes the model parameters use in the experiment. Nonlinear 
thruster dynamics were not included in the vehicle simulation model. 
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Figure 5. 1 Visualization of data point course constraints 



Table 5. 1 Data point course relationships 





15° Solid Angle 


30° Solid Angle 


60° Solid Angle 
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Mirror 
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Table 5.2 Dynamic vehicle simulation parameters 





Extended Mass 
(kg) 


Drag Coefficient 
(kg/m or kg-m) 


Max Thrust 
(N) 


Max Velocity 
(m/s or rad/s) 


X 


1800 


400 


250 


0.25 


y 


1350 


350 


250 


0.50 


Z 


1800 


400 


250 


0.25 


e 


450 


250 


41.5 


0.157 
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Figure 5.2 The world coordinate system 
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Figure 5.3 The vehicle coordinate system 
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5.4 Operators and Training 



The same four operators were used in each of the seven tests. The author was not 
an operator. Operator profiles were: 

Operator I was a 22 year old, male Naval Officer and graduate student with 
uncorrected full visual acuity and normal color vision. This right-handed subject 
had frequently played video games including flight simulators. He was familiar 
with 3-D perspective computer displays prior to the experiment. 

Operator II was a 24 year old, male graduate student with uncorrected full visual 
acuity and normal color vision. This left-handed subject had little exposure to video 
games such as flight simulators. He was familiar with 3-D perspective computer 
displays prior to the experiment due to practical experience with computer systems 
as an undergraduate Computer Science major. 

Operator III was a 34 year old, male Naval Officer and graduate student with 
uncorrected full visual acuity and normal color vision. This right-handed subject 
had some exposure to video games such as flight simulators. He was familiar with 
3-D perspective computer displays prior to the experiment. 

Operator IV was a 32 year old male with a Master’s Degree, uncorrected full visual 
acuity, and normal color vision. This right-handed subject had little exposure to 
video games such as flight simulators. He was not familiar with 3-D perspective 
computer displays prior to the experiment. 

Prior to each testing session, operators were briefed in the following areas: 
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general, simulation, and hardware. The general topics included purpose of thesis and 
experimental approach, time measurement, and operator errors. Simulation topics included 
target and course design as well as vehicle modeling. Hardware subjects discussed impor- 
tant features of the computer monitor, the computer display, and the joystick. Additionally, 
a maximum of 15 minutes was allotted for familiarity training with a 60°, 30°, or 15° field 
of view, as appropriate to the test, using training course X. 

The tests were conducted in a total of four separate sittings to reduce operator 
fatigue. Test 1 (see number designations below) was conducted alone. The remaining 
tests were conducted in pairs in the following order: 2/3, 4/5, and 6/7. 



5.5 Test 1 - Field of View 

Testing for field of view measured changes in the operator’s spatial performance 
versus the displayed field of view. 

Test 1 description : All targets were colored blue. The operators drove the simulated 
vehicle point to point over a course of 25 data points spaced in x, y, and z. Operators 
completed nine courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 
data from point to point. Measured fields of view were 15°, 30°, and 60°. The display 
update period was fixed at two seconds. Nine combinations of FOV and courses were 
used with each of the four operators. 



5.6 Tests 2 and 3 - Gridding 

Testing for gridding measured the improvement in operator performance when 
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presented with a visual reference grid in the display. 



Test 2 description : All targets were colored blue. The operators drove the simulated 
vehicle point to point over a course of 25 data points spaced in x, y, and z. Operators 
completed three courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 
data from point to point. The field of view was fixed at 60° and the display update period at 
two seconds. A two-dimensional (2-D) grid was superimposed on the visual display. 

Test 3 description : All targets were colored blue. The operators drove the simulated 
vehicle point to point over a course of 25 data points spaced in x, y, and z. Operators 
completed three courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 
data from point to point. The field of view was fixed at 60° and the display update period at 
two seconds. A three-dimensional (3-D) grid was superimposed on the visual display. 



5.7 Tests 4 and 5 - Vertical Color Schemes 

Testing for color scheme measured the improvement in operator performance using 
a fixed, vertical-relief color scheme for target shading versus single-color targets. The 6- 
color set provided 12 meter vertical color swaths while the 21 -color set provided three 
meter vertical color swaths over the range of interest. Table 5.3 lists the color sets. 

Test 4 description : Targets were shaded in color according to height z above ocean floor. 
The color scheme was divided into six discrete shades. The operators drove the simulated 
vehicle point to point over a course of 25 data points spaced in x, y, and z. Operators 
completed three courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 
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Table 5.3 



The color sets (1, 6, 21) and associated altitude bands 



Set 


Target 


Color 


Red 


Green 


Blue 


Z value (meters) 


1 


0 


Blue 


0 


0 


1 


All 


6 


1 


Red 


1 


0 


0 


<88 


6 


2 


Yellow 


1 


1 


0 


76-88 


6 


3 


Green 


0 


1 


0 


64-76 


6 


4 


Cyan 


0 


1 


1 


52-64 


6 


5 


Blue 


0 


0 


1 


40-52 


6 


6 


Magenta 


1 


0 


1 


<40 


21 


7 


Red 


1 


0 


0 


>93 


21 


8 


Reddish Orange 


1 


0.25 


0 


90-93 


21 


9 


Orange 


1 


0.50 


0 


87-90 


21 


10 


Yellowish Orange 


1 


0.75 


0 


84-87 


21 


11 


Yellow 


1 


1 


0 


81 - 84 


21 


12 


Yellowish Yellow-Green 


0.75 


1 


0 


78-81 


21 


13 


Yellow-Green 


0.50 


1 


0 


75-78 


21 


14 


Greenish Yellow-Green 


0.25 


1 


0 


72-75 


21 


15 


Green 


0 


1 


0 


69-72 


21 


16 


Greenish Aquamarine 


0 


1 


0.25 


66-69 


21 


17 


Aquamarine 


0 


1 


0.50 


63-66 


21 


18 


Cyanish Aquamarine 


0 


1 


0.75 


60-63 


21 


19 


Cyan 


0 


1 


1 


57-60 


21 


20 


Cyanish Turquoise 


0 


0.75 


1 


54-57 


21 


21 


Turquoise 


0 


0.50 


1 


51-54 


21 


22 


Blueish Turquoise 


0 


0.25 


1 


48-51 


21 


23 


Blue 


0 


0 


1 


45-48 


21 


24 


Blueish Purple 


0.25 


0 


1 


42-45 


21 


25 


Purple 


0.50 


0 


1 


39-42 


21 


26 


Magentaish Purple 


0.75 


0 


1 


36-39 


21 


27 


Magenta 


1 


0 


1 


<36 
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data from point to point. The field of view was fixed at 60° and the display update period at 
two seconds. 



Test 5 description : Targets were shaded in color according to height z above ocean floor. 
The color scheme was divided into 21 discrete shades. The operators drove the simulated 
vehicle point to point over a course of 25 data points spaced in x, y, and z. Operators 
completed three courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 
data from point to point. The field of view was fixed at 60° and the display update period at 
two seconds. 



5.8 Test 6 - Heads Up Range 

Testing for range measured the improvement in operator performance when 
presented with continuously revised range to target center information. 

Test 6 description : All targets were colored blue. The operators drove the simulated 
vehicle point to point over a course of 25 data points spaced in x, y, and z. Operators 
completed three courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 
data from point to point. The field of view was fixed at 60° and the display update period at 
two seconds. Continuously revised range to target center was provided to the operator. 



5.9 Test 7 - Display Update Rate 

Testing for display update rate measured the degradation in operator performance 
associated with reduced visual information feedback. 
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Test 7 description : All targets were colored blue. The operators drove the simulated 
vehicle point to point over a course of 25 data points spaced in x, y, and z. Operators 
completed three courses, each varying in x/y/z with a maximum 15°, 30°, or 60° spread in 



data from point to point. The field of view was fixed at 60°. 
was slowed to four seconds. 



The display update period 
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Chapter 6 



Testing Results 



6.1 General 

The first step in the analysis was to convert the logged target hit times into time 
differences between successive targets. Appendix E lists the resultant time differences. 
The second step was normalize the raw data from Appendix E by removing the effects 
properly attributed to variations in the level of course difficulty related to the Jason vehicle 
simulation model. This procedure divided each time difference (in seconds) by the best 
time (in seconds) that could be made between the associated targets based on the vehicle 
simulation. Appendix D discusses the method of arriving at this best time between targets 
and lists those times in table form. Appendix F contains the normalized, dimensionless 
values. The third step was to estimate the operators' learning curves to standardize results 
between tests. These results were then analyzed directly to assess the effectiveness of the 
tested display enhancements. 



6.2 Operator Learning Curves 

Operator learning reflected a number of factors including increasing familiarity with 
test equipment, improving knowledge of test procedures, and gradual easing of pretest 
anxiety, as well as modified strategies for driving the simulated vehicle. Estimation of the 
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operators' learning curves involved four assumptions. 

• The learning curves had positive slope at all times. This meant that the operators 

were constantly improving with no loss of knowledge or acquired skills 
between testing sessions. 

• The learning curves were smooth functions. This prohibited the consideration of 

learning plateaus during which operators would show no improvement. 

• The learning curves were exponential functions. 

• For a specified field of view, operator learning was reflected primarily in the 

standard deviation of the normalized course times (a smoothness measure). 
The effectiveness of display enhancements was reflected primarily in the 
total course times (a measure of speed). 

As an additional consideration, test 1 was designed to force the operators to use a variety of 
strategies, particularly with the reduced fields of view, in an attempt to ensure maximum 
learning in minimum time. 

Each operator drove 21 courses with a 60° field of view, three for each of the seven 
tests. The procedure below was applied to the normalized data for each operator to estimate 
his 60° FOV learning curve. 

• The standard deviations were entered into a 3 x 7 matrix. Each row represented 

a specific window width P as discussed in section 5.2 (row 1 = 15°, row 2 
= 30°, and row 3 = 60°). Each column corresponded to a given test (column 
1 = test 1, column 2 = test 2, etc.). 

• The standard deviations for the three window widths for each test were summed 

and entered into a row vector of seven elements. Each column continued to 
represent a given test as above. This step yielded the summed standard 
deviations (SSD) shown in table 6.1. 

• The row vector elements were plotted with summed standard deviations on the 

vertical axis versus test number on the horizontal axis. 
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• An exponential curve using the method of least squares was fitted to the plot by 

linearizing the exponential form through taking logarithms [G-l]. This 
defined the operator's learning curve. Appendix G shows the graphed 
operators' learning curves. 

Once the operators’ learning curves were estimated, correction factors to allow 
results between tests to be compared were derived using the following procedure. 

• Estimated standard deviation values were extracted from the fitted exponential 

curve using regression techniques. This step produced the extracted values 
(EV) listed in table 6.1. 

• Standardization factors (SF) were formed by dividing all EVs by the first (and 

largest) value. The EVs are listed in table 6. 1. 

• The 84 mean values of the normalized 60° FOV course times listed in Appendix F 

were multiplied by the associated SF. This step resulted in the learning- 
corrected, normalized mean times (LCNMT) listed in table 6.2. 

• For each test, the LCNMTs for all operators were summed. The mean and 

standard deviation of each LCNMT sum were found, completing the data 
manipulation. 

Figure 6.1 shows the plotted means and standard deviations of the normalized mean times 
without correction for individual operator learning curves. Figure 6.2 shows the means 
and standard deviations of the LCNMT sum plotted for analysis. 

The learning curve for the different fields of view could not be estimated using the 
same techniques. It was assumed that the individual operator's 60° FOV learning curve 
was a good approximation and the first three SFs from table 6.1 for each operator were 
applied to the data. 
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Table 6. 1 Summed standard deviations (SSD), extracted values (EV), and 



standardization factors (SF) 





Operator I 


O 


perator II 


Operator 1 


II 


Operator 1 


[V 


Test 


SSD 


EV 


SF 


SSD 


EV 


SF 


SSD 


EV 


SF 


SSD 


EV 


SF 


1 


4.16 


3.56 


1.00 


7.65 


5.86 


1.00 


7.35 


5.32 


1.00 


3.85 


3.51 


1.00 


2 


3.01 


3.39 


1.05 


5.55 


5.12 


1.15 


3.72 


5.01 


1.06 


3.09 


3.48 


1.01 


3 


3.36 


3.22 


1.11 


4.41 


4.47 


1.31 


5.70 


4.71 


1.13 


3.48 


3.46 


1.01 


4 


2.73 


3.06 


1.17 


2.36 


3.90 


1.50 


3.47 


4.43 


1.20 


3.20 


3.43 


1.02 


5 


2.75 


2.91 


1.23 


2.66 


3.40 


1.72 


3.45 


4.17 


1.28 


3.59 


3.41 


1.03 


6 


2.53 


2.76 


1.29 


3.31 


2.97 


1.97 


3.82 


3.92 


1.36 


3.67 


3.38 


1.04 


7 


3.10 


2.62 


1.36 


3.56 


2.59 


2.26 


4.82 


3.69 


1.44 


3.20 


3.36 


1.04 



Table 6.2 Learning-corrected, normalized mean times (60° FOV) 





Operator I 


O 


perator II 


Operator ] 


II 


Operator ] 


V 


Test 


15° (3 


30° p 


60° p 


15° p 


30° p 


60° p 


15° p 


30° P 


60° p 


15° p 


30° p 


60° p 


1 


1.84 


2.26 


3.62 


3.00 


2.90 


5.55 


2.52 


3.32 


3.85 


3.70 


3.15 


3.48 


2 


2.18 


2.38 


3.65 


2.10 


3.47 


4.72 


2.91 


2.32 


3.77 


2.18 


1.93 


3.03 


3 


1.90 


1.85 


3.26 


3.49 


4.12 


4.65 


3.09 


3.26 


2.81 


1.61 


1.94 


2.95 


4 


1.71 


1.94 


3.09 


2.31 


2.68 


4.57 


2.74 


2.20 


4.20 


1.98 


1.81 


3.11 


5 


2.65 


2.02 


3.43 


3.23 


3.68 


4.78 


2.98 


2.69 


3.11 


1.61 


1.56 


2.93 


6 


1.81 


2.40 


3.61 


3.94 


3.67 


7.77 


2.12 


4.02 


4.68 


1.66 


2.40 


3.14 


7 


2.71 


3.60 


3.97 


6.33 


5.87 


7.81 


3.58 


4.66 


5.63 


2.06 


2.20 


3.40 
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Figure 6.1 Normalized mean times (60° FOV) 
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Figure 6.2 Learning-corrected, normalized mean times (60° FOV) 
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6.3 Field of View 



The three fields of view used in testing were related by inverse powers of two (60°, 
30°, and 15°). Plotted means and standard deviations of the leaming-curve-corrected data 
versus FOV scaled by inverse powers of two were virtually linear and clearly demonstrated 
the parabolic relationship between the mean run time and the FOV. Fitting a parabola to the 
data showed that: 

normalized mean run time = 10.399 - 0.200 * FOV + 1.350 * 10' 3 * FOV^. 



6.4 Gridding 

The 2-D and 3-D grids improved operator performance to the same degree (34.64 
and 34.92 respectively). The 2-D grid was simple and straightforward to employ. The 3- 
D grid conveyed more information, but cluttered the display and was harder to interpret. 
The primary benefit of the 2-D display was the "cross-hairs" effect of the grid, which aided 
steering effectiveness. This effect was less pronounced with the 3-D grid. Hidden-line 
removal, particularly for a 3-D grid, could provide strong visual range cues in the vicinity 
of a target. 



6.5 Vertical Color Schemes 

The 6-color scheme and the 21 -color scheme both provided improved operator 
performance. The 6-color scheme proved to be slightly better (32.34 versus 34.68). This 
effect was attributed to its simplicity and the high-contrast, easily distinguished colors. The 
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Table 6.3 Learning-corrected, normalized mean times (varying FOV) 





Operator I 


O] 


aerator II 


Operator 1 


II 


Operator 1 


V 


FOV 


15° (3 


30° p 


60° p 


15° p 


30° p 


60° P 


15° p 


30° p 


60° p 


15° p 


30° p 


60° p 


60 


1.84 


2.26 


3.62 


3.00 


2.90 


5.55 


2.52 


3.32 


3.85 


3.70 


3.15 


3.48 


30 


5.89 


5.07 


4.99 


4.63 


4.73 


6.05 


6.49 


5.71 


7.13 


3.80 


4.11 


8.83 


15 


5.97 


4.96 


7.17 


7.30 


10.7 


11.6 


2.28 


5.63 


11.1 


6.75 


8.45 


10.5 



mean ± a 




FOV -> 60 30 15 

Figure 6.3 Learning-corrected, normalized mean times (varying FOV) 
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primary benefit of the color schemes was improved spatial perception of the display due to 
the combined effects of depth and altitude cues. These two cues were strongly interrelated. 



6.6 Heads-Up Range 

A surprising result was the degradation of operator performance when supplied 
with range to the target center. Although the range information proved valuable if a target 
was not visible, was close, or had been missed, this was more than offset by negative 
factors. Primary problems included ignoring the visual information of the display and the 
"perigee effect," where a control action that caused the range to initially decrease would not 
necessarily result in a hit. As the vehicle's trajectory passed perigee with the target's 
center, the range increased, frequently confusing the operator. No individual operator’s 
performance improved with range information. 



6.7 Display Update Rate 

As anticipated, slowing the graphics display update rate by a factor of two hurt 
operator performance. The magnitude of this effect, slowing the normalized run time by a 
factor of 1.3, was smaller than anticipated. This had significant implications for field 
applications, where a longer update period would allow the use of less powerful computer 
equipment or more robust programming. The loss of the operator's ability to mentally 
integrate the visual information from the display and use kinetic depth-effect cues was 
mitigated by the more conservative driving approach adopted by the operators. All the 
operators demonstrated this slowing effect with update rate. 
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6.8 Operator Comments 



Relevant operator comments were broken down into the categories of field of view, 
grids, color sets, range data, update rate, test equipment, test design, and strategy 
(operator number is included in parentheses after comments). 

• Field of View- Narrowing the field of view placed a premium on 

maintaining visual contact with the targets (II)- 

• Grids - The "cross-hairs" effect aided steering effectiveness (II, IV). 

Grids provided relative size information (II). 

The 3-D grid was more intuitive and effective than the 2-D 
grid (II). 

The 3-D grid cluttered the display (IV). 

The grids were not particularly helpful (III). 

Hidden-line removal for grids could provide strong visual 
range cues in the immediate vicinity of a target by 
showing the target "poking through" the grid (I, IV). 
The black grids were sometimes confused with the blue 
targets when the display was cluttered (IV). 

The operator's need for a grid was reduced by improvements 
in his spatial perception (IV). 

• Color Sets - Color provided spatial visualization cues, especially with 

overlapping targets (all). 

The 21-color set provided significandy more information and 
was more effective than the 6-color set (II, IV). 

Color information was far superior to gridding (III). 

Target course altitude slope information was available from 
the rate of change of target colors (III). 
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Range Data - 



Update Rate - 



Test Equipment - 



Test Design - 



A sliding color scale with the vehicle always at scale center 
could provide improved relative altitude cues (III). 

Range information proved valuable if a target was not 

visible, was close, or had been missed (all). 

Range information was ignored if a target was in view (II). 

Range information provided for smoother driving near 

targets and eased hitting targets off-center (II, IV). 

The operator perceived that targets were "hit" from a greater 
distance with range information available (111). 

Operators drove more conservatively with the slower update 
rate (II, III). 

Update rate was more important as the range decreased (III). 

Vehicle alignment with the target trend was more important 
with the slower update rate due to the time lag in 
update rate while rotating (IV). 

Operator workload increased with slower update rate (I). 

The slow update rate was not as difficult as anticipated (TV). 

The joystick was very tiring to use, particularly the z-axis 
pure-force control (III). 

Fine z-axis movement was difficult due to the stiff pure- 
force control (II, III). 

Axis movement with the joystick was not compatible with 
operator expectations (I, III). 

Presentation of all information on one screen could ease 
operator workload (all). 

Target drawing order proved confusing (I, II, III). 

The display graphics can be cluttered, particularly with a 



narrow field of view (I, II, III). 



The simulated vehicle lacked an effective altitude search 
capability (I, III). 

Operator efficiency was reduced by the fixed 2-D target 
numbers when approaching targets from the side 
(all). 

Improved heading perception was provided by the fixed 2-D 
target numbers (II). 

• Strategy - Operator comfort with vehicle rotation improved as testing 

progressed (I, III). 

Operators needed to plan more than one target in advance 
and, when possible, kept them on the display (II, 
III). 

Rotating the vehicle to line up along the target heading trend 
reduced the time between targets (II, HI). 

Knowledge of the course trend aided target identification 
(IV). 

Driving while backing up was counterintuitive (III). 



6.9 Test-Monitor Observations 

Test-monitor observations not previously listed under operator comments were 
broken down into the categories of color sets, range data, update rate, test equipment, test 
design, strategy, and tendencies. 

• Color Sets - Some colors were weak in contrast with the background 
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Range Data - 



Update Rate - 



Test Equipment - 



Test Design - 



Strategy - 



dark gray. Reddish Orange and Turquoise were 
most difficult for the operators to discern. 

Range information sometimes handicapped the operators by 
providing too much data, causing them to ignore the 
display visual information. Another detrimental 
effect was that a control action that caused the 
range to initially decrease would not necessarily 
result in a hit As the vehicle's trajectory passed 
perigee with the target’s center, the range would 
increase. This effect fooled all operators initially, 
particularly when moving the vehicle along more 
than one axis simultaneously. 

The four-second update period significantly degraded the 
operator's ability to mentally integrate the display 
information and use kinetic depth-effect cues. 

Stiffness of the joystick resulted in unorthodox grips being 
employed, many with both hands. 

An artifact of the wireframe target design was that the 
simulated vehicle could be close to or within the 
target without any portion of the target being visible 
on the display. This is an unlikely scenario in the 
field. 

Backing up to keep the previous target in view while looking 
for the next target was an effective approach when 
presented with a limited field of view. 

Heading information was underutilized. Two possible uses 
were to measure the target trend heading and align 



• Tendencies - 



the vehicle accordingly, and to improve the general 
sense of spatial orientation of the operator. 

Operators adopted differing strategies for driving. These 
strategies were modified during testing but did not 
converge to a single "correct" method. 

An effective search technique when no target was in view 
was critical in reducing course run time. 

"Crabbing" or twisting one direction while moving side to 
side in the opposite direction was effective in 
realigning with a new course heading trend while 
keeping targets in view. 

Applying excessive control actions and failing to account for 
the delay caused by the update rate was a problem, 
especially when in close proximity to a target. 

Operators were hesitant to identify target numbers when 

only a portion of the number was visible or the angle 
of view approached 90°. 

As testing progressed, the operators became more confident 
in controlling the vehicle in two or three directions 
simultaneously. 

Operators occasionally applied the correct axis-control order 
in the opposite direction from that intended. This 
was related to the well-known outside-in versus 
inside-out viewing problem. 

Operator average velocities increased as familiarity with the 
testing improved. 
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Chapter 7 



Summary, Conclusions and Recommendations 



7.1 Summary 

Effective collection, computation, and presentation of undersea data to the ROV 
pilot on a single screen presents hardware, software and human-factors problems. Pilot 
performance efficiency through improved display design is a key human factors issue. 
Seven tests measured the effects of an enhanced 3-D perspective display on the simulated 
piloting of an ROV. The effects of the ROV simulation and operator learning curves were 
removed to directly compare performance changes due to the various enhancements. 



7.2 Conclusions 

Display enhancements that improved operator performance were: 

• widening the field of view. 

• providing a gridded screen reference. 

• using vertical color schemes to cue depth/altitude information. 

• shortening the display update period. 

The singularly most important factor for the operator was widening the field of 
view. Providing the operator with continuous range information to the target center 
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degraded performance. 



Many operator comments carried strong implications for system design. 

• The "cross-hairs" effect of a grid aided steering effectiveness. 

• Hidden-line removal for grids could provide strong visual-range cues in the 

immediate vicinity of a target by showing the target "poking through" the 
grid. 

• Color provided spatial visualization cues and altitude slope information. 

• Range information proved valuable if a target was not visible, was close, or had 

been missed. 

• The update rate was more important as the range decreased. 

• The joystick was very tiring to use. Fine z-axis movement was difficult due to 

the stiff pure-force control. 

• Presentation of all information on one screen could ease operator workload. 

Several operator comments were not supported by the test results. 

• The 3-D grid was not more effective than the 2-D grid. They were nearly 

identical in overall effectiveness. 

• The color schemes were not far superior to the grids. The difference between the 

2-D grid, the 3-D grid, and the 21 -color scheme was insignificant. The 6 
color scheme was only slightly better. 

• The 21 -color set was not more effective than the 6-color set. 

Several significant improvements to the proposed 3-D perspective display are 
available through incorporation of the best features of the display enhancements tested. A 
single screen with as wide a field of view as possible and limited distortion, a cross-hair, a 
simple vertical color scheme, target range information, and a reasonable update period (two 
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seconds or less) would improve ROV pilot effectiveness. More testing would be necessary 
to determine the best design for a 2-D or 3-D grid. The key to effective use of range 
information is the method of employment. 



7.3 Recommendations 

This thesis investigated the effects of individual display enhancements on operator 
performance. In order to take maximum advantage of the positive aspects that each of the 
enhancement has to offer, several need to be implemented simultaneously. Due to the 
potentially complex nature of the enhancement interactions, it is unwise to generalize the 
individual results into group results. These interactions need to be analyzed on a case 
basis. 

The presentation of both printed information such as position and velocities as well 
as graphics on the same screen was recommended by all operators. This capability exists 
in modem computer windowing environments. The effectiveness of any proposed screen 
layout design requires both qualitative and quantitative assessment. 

One operator commented that a sliding color scale with the vehicle always at scale 
center could provide improved relative-altitude cues. This suggestion has merit and bears 
further investigation. 

Both operators and pilots expressed strong dissatisfaction with the Measurement 
Systems, Inc., joystick designs (fourth-axis control by lifting/depressing the stick vertically 
or pure-force button). The sticks are far too difficult to use for extended periods due to 
their bulk, stiffness, and awkward incorporation of fourth-axis control. Catalog search has 
yet to produce a fully acceptable alternative, although a different joystick model is currently 
in use. Pilot preferences indicate that a modified video-game joystick is closer to their 
concept of an acceptable control. Identification of design requirements, development, 
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testing to ensure effectiveness, and procurement of a fully acceptable control is an area 
needing attention. 

Between testing sessions, usually a day or two apart, operators experienced a decay 
of previously acquired skills. The driving of course X prior to each test appeared to 
reverse this decay. ROV piloting is not a full-time occupation. Frequently months pass 
between cruises. Pilots also change employers, reducing the corporate piloting knowledge 
level. Three actions are recommended to reduce this loss of skills: 

• Holding precruise pilot discussions to reawaken piloting skills. Critical to these 

discussions would be the minutes of the previous post-cruise debriefs. 

• Conduct of precruise pilot simulator training. Construction of a simulator with 

the field equipment would be straightforward and all the necessary hardware 
is already in hand. Pilot discussions of strategies could be demonstrated 
and tested prior to use in the field. Added benefits of a simulator include 
serving as a test bed for new display designs and the ability to write the 
simulator software to incorporate precruise system checks. 

• Holding postcruise debriefs to document problems, fixes, and strategies. The 

minutes of these debriefs would generate a valuable storehouse of piloting 
knowledge. 
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APPENDIX A 



Nomenclature 



Symbol 


Meaning 


Units 


Page 


C D 


coefficient of drag 


kg/m or kg-m 


38 


4 


look direction matrix from point Pj to 








point Pj + i in vehicle system 


none 


37 


(Lj)* 


transformation matrix from vehicle 








system to world system 


none 


37 


M 


vehicle mass 


kg 


38 


M a 


vehicle added mass 


kg 


38 


M k 


rotation matrix for K angular rotation 


none 


37 


Mq> 


rotation matrix for <p angular rotation 


none 


37 


Mco 


rotation matrix for (0 angular rotation 


none 


37 


P J 


j-th data point 


none 


35 


T 


maximum thrust 


N 


38 


P 


operator field of view angular width 


rad 


35 


0 


vehicle system rotation angle about z axis 


rad 


26 


K 


world system rotation angle about z axis 


rad 


36 


a 


standard deviation 


none 


51 


%iax 


linear drag (damping) coefficient 


s 


38 


<P 


world system rotation angle about y axis 


rad 


36 


(0 


world system rotation angle about x axis 


rad 


36 
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kg kilograms 



m meters 

N newtons 

rad radians 

s seconds 
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APPENDIX B 



Abbreviations 



Symbol 


Meaning 


Page 


ACM 


Association for Computing Machinery 


133 


BBN 


Bolt, Beranek, and Newman 


23 


CCW 


Counterclockwise 


38 


CDR 


Commander 


3 


cm 


Commission Internationale de l'Eclairage 


19 


CRT 


Cathode Ray Tube 


13 


cw 


Clockwise 


38 


DSL 


Deep Submergence Laboratory 


8 


DV 


Dependent Variable 


22 


EV 


Extracted Value 


10 


FJCC 


Fall Joint Computer Congress 


134 


FOV 


Field of View 


8 


GDM 


Graphic Display Monitor 


27 


HOOPS 


Hierarchical Object Oriented Picture System 


25 


HSB 


Hue, Saturation, and Brightness 


10 


IEEE 


Institute of Electrical and Electronics Engineers 


133 


IFIP 


International Federation for Information Processing 


134 


ISCC-NBS 


Inter-Society Color Council - National Bureau of Standards 


19 


IV 


Independent Variable 


22 


LCDR 


Lieutenant Commander 


136 
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LCNMT 


Learning-Corrected, Normalized Mean Times 


49 


MIPS 


Millions of Instructions Per Second 


26 


MIT 


Massachusetts Institute of Technology 


1 


MFLOPS 


Millions of Floating-Point Operations Per Second 


26 


N/A 


Not Applicable 


73 


NEC 


Nippon Electric Company 


27 


NTSC 


National Television System Committee 


26 


RGB 


Red, Green, Blue 


8 


ROV 


Remotely Operated Vehicle 


2 


SF 


Standardization Factor 


10 


SHARPS 


Sonic High Accuracy Ranging and Positioning System 


31 


SONAR 


Sound Navigation and Ranging 


2 


SSD 


Summed Standard Deviation 


10 


n 


Texas Instruments 


23 


UCL 


Universal Color Language 


19 


USN 


United States Navy 


3 


VCR 


Video Cassette Recorder 


26 


VGA 


Video Graphics Adapter 


27 


WHOI 


Woods Hole Oceanographic Institution 


1 


YIQ 


Luminance, In-phase, and Quadrature 


19 


2-D 


Two-Dimensional 


8 


3-D 


Three-Dimensional 


2 
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APPENDIX C 



Target and Grid Design 



All seven tests used the same 25 cubic-wireframe targets. Each target was one 
meter on a side with the invisible x, y, and z data point at its center. Target faces were 
parallel to the planes formed by the world coordinate system axes. Target colors were blue 
except for tests 5 and 6 as discussed in section 5.7. Each target was numbered with white 
numerals (1-25) corresponding to the sequential number of the data point at its center. The 
numbers were 2-D wireframes 0.5 meters in height, 0.3 meters in width (numeral one was 
0.1 meters in width), and drawn parallel to the XZ world coordinate plane facing the initial 
position of the vehicle. All numbers were drawn centered in the target in the x, y, and z 
directions. 

Embedded in each target was an invisible sphere of one-meter diameter. This 
sphere defined the "hit" criterion for its target. When the vehicle, modeled as a point, 
touched the sphere, the target was hit This corresponded to closing the range to the target 
center to 0.5 meters or less from any direction. 

The 2-D grid used in test 2 was composed of 16 black squares arranged in four 
rows of four. The grid was drawn centered, parallel to, and aligned with the edges of the 
display face at a screen depth of five meters. Each square was scaled so it was one meter 
on a side at that depth. Figure C.l shows the operator's display of the 2-D grid with target 
course X. 

The 3-D grid used in test 3 was composed of 200 black squares and rectangles 
arranged in three panels. The center panel was made up of 100 squares in 10 rows of 10. 
This grid was drawn centered, parallel to, and aligned with the edges of the display face at 
a screen depth of five meters. Each square was scaled so that it was 0.25 meter on a side at 
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that depth. The side panels were each made up of 50 rectangles in 10 rows of five. Each 
rectangle was 0.25 meter in height and one meter in width. The side panels were attached 
to the center grid at right angles to its left and right edges. The assembled grid formed a 
box with no top or bottom facing the operator. Figure C.2 shows the operator’s display of 
the 3-D grid with target course X. 
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Figure C. 1 2-D grid (operator's display with 60° FOV and target course X) 




Figure C.2 3-D grid (operator’s display with 60° FOV and target course X) 
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APPENDIX D 



Data Point Courses 

Tables D. 1 through D. 19 list the x, y, and z positions of the centers of targets 1 
through 25 for each of the target courses. The method for generating these x, y, and z 
positions is discussed in detail in section 5.2. Additionally, the shortest time difference 
(best 8t ) between the centers of successive targets is listed for courses A through R. The 
best 5t between targets U and V was calculated as follows: 

• altitude distance from z u to z v was calculated, then divided by the best up/down 

speed (0.25 m/s), resulting in the best up/down time. 

• the change in vehicle heading at target U to fly directly to target V was noted. 

• distance in the horizontal plane from ( x u , y u ) to ( x v , y v ) was calculated. 

• this horizontal distance was decomposed into forward and lateral components 

based on the vehicle's heading prior to reaching target U. 

• the decomposed forward distance was divided by the best forward speed (0.5 

m/s). 

• the decomposed lateral distance was divided by the best lateral speed (0.25 m/s). 

• the minimum time to travel the horizontal distance without heading change was 

the larger of the best forward and lateral speed times. 

• vehicle heading change was divided by the best rotational velocity (0.157 rad/s). 

• the total horizontal distance was divided by the best forward speed (0.5 m/s). 

• the minimum time to travel the horizontal distance with heading change was the 

sum of the best rotational velocity time and the best forward speed time. 

• the minimum time to travel the horizontal distance was the smaller of the times 

with and without heading change. 
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• the best time between targets was the larger of the best up/down and horizontal 
times. 

Figures D.l through D.18 were reproduced from photographs of the operator's 
initial display for courses A through R with a 60° field of view. Figures D.19, D.20, and 
D.21 were reproductions of the initial display for course X with a 60°, 30°, and 15° field of 
view respectively. 
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Table D.l 



Course A 



Target 


X 


JL_ 


z 


Best 5t 


1 


75 


5 


50 


N/A 


2 


75.340 


9.971 


50.419 


9.942 


3 


75.683 


14.959 


50.393 


10.000 


4 


75.743 


19.937 


50.862 


9.940 


5 


76.337 


24.897 


51.060 


9.935 


6 


77.204 


29.809 


50.703 


9.959 


7 


78.401 


34.662 


50.821 


9.975 


8 


79.848 


39.447 


50.718 


9.984 


9 


80.992 


44.314 


50.665 


9.980 


10 


82.121 


49.184 


50.707 


10.000 


11 


83.810 


53.888 


50.570 


9.928 


12 


86.077 


58.330 


50.930 


9.894 


13 


88.094 


62.885 


51.366 


9.947 


14 


90.093 


67.411 


52.085 


9.896 


15 


91.666 


72.019 


53.219 


9.704 


16 


92.761 


76.807 


54.158 


9.770 


17 


93.676 


81.488 


55.659 


9.536 


18 


94.076 


86.374 


56.640 


9.742 


19 


94.950 


91.283 


57.016 


9.935 


20 


95.347 


96.261 


57.262 


9.940 


21 


95.595 


101.250 


57.047 


9.984 


22 


95.454 


106.237 


56.713 


9.947 


23 


95.215 


111.159 


55.866 


9.852 


24 


94.726 


115.948 


54.514 


9.612 


25 


94.792 


120.760 


53.159 


9.566 
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Table D.2 



Course B 



Target 


X 


y 


z 


Best 5t 


1 


75 


5 


50 


N/A 


2 


75.514 


9.942 


50.561 


9.884 


3 


76.280 


14.801 


51.459 


9.825 


4 


76.464 


19.731 


52.270 


9.800 


5 


76.788 


24.651 


53.100 


9.857 


6 


77.619 


29.517 


53.893 


9.822 


7 


78.714 


34.369 


54.406 


9.933 


8 


80.191 


39.111 


54.981 


9.903 


9 


82.247 


43.558 


55.978 


9.723 


10 


83.945 


48.008 


57.501 


9.500 


11 


85.987 


52.139 


59.440 


9.186 


12 


88.368 


56.088 


61.373 


9.196 


13 


90.182 


60.266 


63.436 


9.039 


14 


91.667 


64.362 


65.889 


9.812 


15 


92.799 


68.246 


68.827 


11.755 


16 


93.692 


72.084 


71.905 


12.311 


17 


94.672 


75.833 


75.064 


12.636 


18 


96.247 


79.760 


77.729 


10.659 


19 


97.585 


83.600 


80.639 


11.639 


20 


98.424 


87.377 


83.806 


12.669 


21 


99.684 


90.714 


87.309 


14.012 


22 


101.492 


93.759 


90.839 


14.118 


23 


102.575 


96.682 


94.748 


15.637 


24 


103.974 


99.466 


98.659 


15.643 


25 


105.626 


101.705 


102.813 


16.618 
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Figure D. 1 Course A (operator’s display with 60° FOV) 




Figure D.2 Course B (operator's display with 60° FOV) 
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Table D.3 Course C 



Target 


X 


y 


z 


Best 8t 


1 


75 


5 


50 


N/A 


2 


75.020 


9.983 


50.415 


9.965 


3 


74.980 


14.977 


50.180 


9.988 


4 


74.925 


19.976 


50.095 


9.999 


5 


75.269 


24.919 


49.429 


9.879 


6 


75.592 


29.763 


48.232 


9.709 


7 


76.443 


34.610 


47.343 


9.786 


8 


77.675 


39.396 


46.586 


9.855 


9 


78.482 


44.192 


45.423 


9.690 


10 


79.342 


48.955 


44.170 


9.680 


11 


80.071 


53.561 


42.368 


9.325 


12 


80.670 


58.230 


40.681 


9.410 


13 


81.492 


63.000 


39.427 


9.672 


14 


82.575 


67.842 


38.810 


9.911 


15 


83.997 


72.636 


38.769 


9.974 


16 


85.192 


77.489 


38.898 


9.986 


17 


86.296 


82.364 


39.015 


9.995 


18 


87.120 


87.270 


38.511 


9.932 


19 


88.257 


92.015 


37.420 


9.738 


20 


89.117 


96.742 


36.035 


9.594 


21 


89.412 


101.410 


34.268 


9.297 


22 


89.760 


106.245 


33.043 


9.695 


23 


89.707 


111.017 


31.552 


9.516 


24 


89.728 


115.746 


29.930 


9.458 


25 


89.851 


120.359 


28.004 


9.226 
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Table D.4 



Course D 



Target 


X 


y 


z 


Best 8t 


1 


75 


5 


50 


N/A 


2 


74.372 


9.917 


50.658 


9.835 


3 


74.614 


14.903 


50.928 


9.835 


4 


74.826 


19.671 


52.420 


9.544 


5 


75.283 


24.294 


54.270 


9.278 


6 


76.893 


28.957 


55.080 


9.601 


7 


79.632 


33.111 


55.573 


9.637 


8 


82.281 


37.123 


56.946 


9.615 


9 


85.192 


41.067 


57.934 


9.787 


10 


87.250 


45.189 


59.876 


9.073 


11 


89.045 


48.851 


62.768 


11.570 


12 


91.190 


52.496 


65.436 


10.670 


13 


93.106 


56.769 


67.187 


9.327 


14 


94.735 


61.062 


69.167 


9.169 


15 


95.491 


65.646 


71.015 


9.134 


16 


95.769 


70.151 


73.167 


8.993 


17 


95.051 


74.059 


76.202 


12.139 


18 


94.195 


78.131 


78.975 


11.092 


19 


94.587 


82.593 


81.196 


8.885 


20 


95.963 


86.945 


83.236 


8.962 


21 


96.694 


91.569 


84.993 


9.265 


22 


98.573 


96.153 


85.668 


9.709 


23 


100.225 


100.734 


86.802 


9.734 


24 


101.228 


105.627 


86.583 


9.845 


25 


101.964 


110.571 


86.428 


9.981 
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Figure D.3 Course C (operator's display with 60° FOV) 




Figure D.4 Course D (operator's display with 60° FOV) 
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Table D.5 



Course E 



Target 


X 


y 


z 


Best 5t 


1 


75 


5 


50 


N/A 


2 


74.164 


9.874 


50.737 


9.752 


3 


72.990 


14.566 


52.005 


9.649 


4 


72.104 


19.139 


53.823 


9.301 


5 


70.407 


23.580 


55.370 


9.378 


6 


68.214 


28.060 


55.718 


9.917 


7 


66.758 


32.567 


57.322 


9.357 


8 


64.791 


36.764 


59.197 


9.209 


9 


63.625 


41.499 


60.302 


9.599 


10 


61.257 


45.902 


60.362 


9.663 


11 


59.506 


50.584 


60.486 


9.905 


12 


58.125 


55.374 


60.874 


9.939 


13 


57.941 


60.311 


61.644 


9.591 


14 


58.232 


65.298 


61.853 


9.945 


15 


59.369 


70.150 


62.256 


9.822 


16 


60.703 


74.966 


62.424 


9.986 


17 


63.211 


79.278 


62.770 


9.653 


18 


65.725 


83.598 


62.902 


9.997 


19 


68.894 


87.449 


63.255 


9.849 


20 


71.096 


91.933 


63.470 


9.728 


21 


73.080 


96.326 


64.797 


9.633 


22 


75.684 


100.527 


65.557 


9.796 


23 


77.857 


105.028 


65.683 


9.949 


24 


79.568 


109.626 


64.715 


9.780 


25 


81.847 


113.963 


63.718 


9.723 



79 



Table D.6 



Course F 



Target 


X 


y 


z 


Best 8t 


1 


75 


5 


50 


N/A 


2 


74.090 


9.822 


50.957 


9.651 


3 


72.789 


14.460 


52.299 


9.602 


4 


71.500 


18.826 


54.367 


9.105 


5 


71.318 


23.509 


56.109 


9.130 


6 


71.669 


28.250 


57.657 


9.453 


7 


70.716 


32.909 


59.205 


9.185 


8 


69.282 


37.669 


59.737 


9.897 


9 


67.473 


42.322 


60.019 


9.952 


10 


65.537 


46.929 


60.170 


9.991 


11 


63.072 


51.277 


60.311 


9.927 


12 


61.000 


55.811 


59.923 


9.936 


13 


58.565 


60.063 


58.930 


9.756 


14 


56.794 


64.739 


58.944 


9.864 


15 


53.946 


68.755 


58.072 


9.523 


16 


51.351 


73.023 


57.847 


9.966 


17 


49.014 


77.376 


57.080 


9.867 


18 


45.742 


80.745 


55.364 


9.074 


19 


41.720 


83.431 


54.096 


9.457 


20 


38.238 


87.010 


53.838 


9.831 


21 


34.887 


90.688 


54.334 


9.950 


22 


31.512 


94.286 


53.521 


9.864 


23 


29.020 


98.614 


53.754 


9.792 


24 


27.165 


103.247 


54.071 


9.886 


25 


25.496 


107.871 


53.158 


9.806 
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Figure D.5 Course E (operator's display with 60° FOV) 




Figure D.6 Course F (operator's display with 60° FOV) 
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Table D.7 



Course G 



Target 


X 


y 


z 


Best 5t 


1 


75 


5 


50 


N/A 


2 


77.402 


9.372 


49.658 


9.584 


3 


80.930 


12.001 


47.283 


9.501 


4 


85.722 


13.316 


46.731 


8.949 


5 


90.100 


13.069 


44.328 


9.609 


6 


95.045 


13.750 


44.040 


9.983 


7 


99.803 


12.432 


43.248 


9.344 


8 


104.761 


12.926 


42.829 


9.532 


9 


109.181 


12.034 


44.988 


8.637 


10 


111.078 


10.074 


49.179 


16.763 


11 


110.555 


7.725 


53.562 


17.531 


12 


110.822 


7.113 


58.517 


19.820 


13 


112.792 


8.030 


63.020 


18.011 


14 


115.272 


10.445 


66.628 


14.434 


15 


116.622 


14.175 


69.671 


12.174 


16 


118.163 


16.656 


73.730 


16.233 


17 


117.290 


18.362 


78.348 


18.473 


18 


118.453 


19.258 


83.127 


19.118 


19 


120.460 


18.207 


87.585 


17.829 


20 


123.046 


15.938 


91.212 


14.510 


21 


126.242 


14.134 


94.609 


13.584 


22 


130.759 


13.240 


96.559 


8.860 


23 


135.067 


14.923 


98.456 


9.114 


24 


139.759 


16.585 


98.932 


9.935 


25 


144.021 


18.058 


101.093 


8.962 
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Table D.8 



Course H 



Target 


X 


y 


z 


Best 8t 


1 


75 


5 


50 


N/A 


2 


73.035 


9.488 


50.999 


9.010 


3 


71.086 


14.085 


51.257 


9.987 


4 


69.943 


18.154 


53.928 


10.684 


5 


68.558 


21.353 


57.512 


14.338 


6 


65.215 


23.831 


60.286 


11.093 


7 


60.466 


25.325 


60.742 


9.566 


8 


55.907 


26.833 


62.137 


9.558 


9 


52.630 


30.596 


62.454 


9.227 


10 


52.017 


35.556 


62.305 


9.426 


11 


51.968 


40.348 


63.733 


9.281 


12 


50.143 


44.961 


64.353 


9.166 


13 


46.707 


48.593 


64.334 


9.257 


14 


43.809 


52.163 


66.298 


8.822 


15 


40.102 


55.081 


67.954 


9.230 


16 


36.661 


57.474 


70.681 


10.907 


17 


33.390 


57.996 


74.426 


14.981 


18 


31.969 


58.438 


79.200 


19.094 


19 


29.845 


57.500 


83.628 


17.712 


20 


29.339 


57.006 


88.577 


19.799 


21 


26.730 


57.726 


92.782 


16.818 


22 


22.486 


57.380 


95.403 


10.483 


23 


18.256 


58.209 


97.936 


10.135 


24 


14.084 


60.827 


98.797 


9.851 


25 


9.687 


62.529 


100.460 


9.427 
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Figure D.7 



Figure D.8 




Course G (operator's display with 60° FOV) 




Course H (operator's display with 60° FOV) 
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Table D.9 



Course I 



Target 


X 


y 


z 


Best 8t 


1 


75 


5 


50 


N/A 


2 


73.705 


9.827 


49.845 


9.654 


3 


70.962 


13.348 


47.593 


9.011 


4 


69.007 


17.733 


46.196 


9.422 


5 


65.232 


20.735 


44.877 


8.664 


6 


61.594 


24.027 


43.913 


9.805 


7 


59.815 


28.690 


43.604 


9.113 


8 


59.727 


33.487 


45.011 


9.341 


9 


58.587 


38.275 


45.893 


9.711 


10 


55.693 


41.610 


48.238 


9.379 


11 


53.396 


44.963 


51.151 


11.651 


12 


50.001 


45.848 


54.713 


14.250 


13 


47.180 


43.768 


58.279 


14.261 


14 


46.505 


40.286 


61.803 


14.096 


15 


45.997 


37.783 


66.101 


17.195 


16 


45.976 


34.940 


70.214 


16.450 


17 


45.205 


32.056 


74.225 


16.044 


18 


44.872 


28.706 


77.922 


14.787 


19 


42.788 


24.785 


80.222 


9.200 


20 


39.883 


20.716 


80.238 


8.983 


21 


35.754 


18.089 


81.262 


9.452 


22 


31.130 


17.307 


79.527 


8.462 


23 


26.277 


16.223 


80.051 


9.048 


24 


21.766 


15.832 


77.931 


8.483 


25 


17.182 


17.512 


76.854 


9.276 
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Table D. 10 Course J 



Target 


X 


y 


z 


Best 8t 


1 


75 


5 


50 


N/A 


2 


74.660 


9.971 


50.419 


9.942 


3 


74.317 


14.959 


50.393 


10.000 


4 


74.257 


19.937 


50.862 


9.940 


5 


73.663 


24.897 


51.060 


9.935 


6 


72.796 


29.809 


50.703 


9.959 


7 


71.599 


34.662 


50.821 


9.975 


8 


70.152 


39.447 


50.718 


9.984 


9 


69.008 


44.314 


50.665 


9.980 


10 


67.879 


49.184 


50.707 


10.000 


11 


66.190 


53.888 


50.570 


9.928 


12 


63.923 


58.330 


50.930 


9.894 


13 


61.906 


62.885 


51.366 


9.947 


14 


59.907 


67.411 


52.085 


9.896 


15 


58.334 


72.019 


53.219 


9.704 


16 


57.239 


76.807 


54.158 


9.770 


17 


56.324 


81.488 


55.659 


9.536 


18 


55.924 


86.374 


56.640 


9.742 


19 


55.050 


91.283 


57.016 


9.935 


20 


54.653 


96.261 


57.262 


9.940 


21 


54.405 


101.250 


57.047 


9.984 


22 


54.546 


106.237 


56.713 


9.947 


23 


54.785 


111.159 


55.866 


9.852 


24 


55.274 


115.948 


54.514 


9.612 


25 


55.208 


120.760 


53.159 


9.566 
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Figure D.9 Course I (operator's display with 60° FOV) 




Figure D.10 Course J (operator's display with 60° FOV) 
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Table D. 1 1 Course K 



Target 


X 


y 


z 


Best 8t 


1 


75 


5 


50 


N/A 


2 


74.486 


9.942 


50.561 


9.884 


3 


73.720 


14.801 


51.459 


9.825 


4 


73.536 


19.731 


52.270 


9.800 


5 


73.212 


24.651 


53.100 


9.857 


6 


72.381 


29.517 


53.893 


9.822 


7 


71.286 


34.369 


54.406 


9.933 


8 


69.809 


39.111 


54.981 


9.903 


9 


67.753 


43.558 


55.978 


9.723 


10 


66.055 


48.008 


57.501 


9.500 


11 


64.013 


52.139 


59.440 


9.186 


12 


61.632 


56.088 


61.373 


9.196 


13 


59.818 


60.266 


63.436 


9.039 


14 


58.333 


64.362 


65.889 


9.812 


15 


57.201 


68.246 


68.827 


11.755 


16 


56.308 


72.084 


71.905 


12.311 


17 


55.328 


75.833 


75.064 


12.636 


18 


53.753 


79.760 


77.729 


10.659 


19 


52.415 


83.600 


80.639 


11.639 


20 


51.576 


87.377 


83.806 


12.669 


21 


50.316 


90.714 


87.309 


14.012 


22 


48.508 


93.759 


90.839 


14.118 


23 


47.425 


96.682 


94.748 


15.637 


24 


46.026 


99.466 


98.659 


15.643 


25 


44.374 


101.705 


102.813 


16.618 
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Table D. 1 2 Course L 



Target 


X 


_2 


z 


Best 8t 


1 


75 


5 


50 


N/A 


2 


74.980 


9.983 


50.415 


9.965 


3 


75.020 


14.977 


50.180 


9.988 


4 


75.075 


19.976 


50.095 


9.999 


5 


74.731 


24.919 


49.429 


9.879 


6 


74.408 


29.763 


48.232 


9.709 


7 


73.557 


34.610 


47.343 


9.786 


8 


72.325 


39.396 


46.586 


9.855 


9 


71.518 


44.192 


45.423 


9.690 


10 


70.658 


48.955 


44.170 


9.680 


11 


69.929 


53.561 


42.368 


9.325 


12 


69.330 


58.230 


40.681 


9.410 


13 


68.508 


63.000 


39.427 


9.672 


14 


67.425 


67.842 


38.810 


9.911 


15 


66.003 


72.636 


38.769 


9.974 


16 


64.808 


77.489 


38.898 


9.986 


17 


63.704 


82.364 


39.015 


9.995 


18 


62.880 


87.270 


38.511 


9.932 


19 


61.743 


92.015 


37.420 


9.738 


20 


60.883 


96.742 


36.035 


9.594 


21 


60.588 


101.410 


34.268 


9.297 


22 


60.240 


106.245 


33.043 


9.695 


23 


60.293 


111.017 


31.552 


9.516 


24 


60.272 


1 15.746 


29.930 


9.458 


25 


60.149 


120.359 


28.004 


9.226 
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Figure D. 1 1 



Figure D.12 




Course K (operator's display with 60° FOV) 




Course L (operator's display with 60° FOV) 
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Table D. 1 3 Course M 



Target 


X 


y 


z 


Best 8t 


1 


75 


5 


50 


N/A 


2 


75.628 


9.917 


50.658 


9.835 


3 


75.386 


14.903 


50.928 


9.835 


4 


75.174 


19.671 


52.420 


9.544 


5 


74.717 


24.294 


54.270 


9.278 


6 


73.107 


28.957 


55.080 


9.601 


7 


70.368 


33.111 


55.573 


9.637 


8 


67.719 


37.123 


56.946 


9.615 


9 


64.808 


41.067 


57.934 


9.787 


10 


62.750 


45.189 


59.876 


9.073 


11 


60.955 


48.851 


62.768 


11.570 


12 


58.810 


52.496 


65.436 


10.670 


13 


56.894 


56.769 


67.187 


9.327 


14 


55.265 


61.062 


69.167 


9.169 


15 


54.509 


.65.646 


71.015 


9.134 


16 


54.231 


70.151 


73.167 


8.993 


17 


54.949 


74.059 


76.202 


12.139 


18 


55.805 


78.131 


78.975 


11.092 


19 


55.413 


82.593 


81.196 


8.885 


20 


54.037 


86.945 


83.236 


8.962 


21 


53.306 


91.569 


84.993 


9.265 


22 


51.427 


96.153 


85.668 


9.709 


23 


49.775 


100.734 


86.802 


9.734 


24 


48.772 


105.627 


86.583 


9.845 


25 


48.036 


110.571 


86.428 


9.981 



91 




Table D.14 Course N 



Target 


X 


y 


z 


Best 5t 


1 


75 


5 


50 


N/A 


2 


75.836 


9.874 


50.737 


9.752 


3 


77.010 


14.566 


52.005 


9.649 


4 


77.896 


19.139 


53.823 


9.301 


5 


79.593 


23.580 


55.370 


9.378 


6 


81.786 


28.060 


55.718 


9.917 


7 


83.242 


32.567 


57.322 


9.357 


8 


85.209 


36.764 


59.197 


9.209 


9 


86.375 


41.499 


60.302 


9.599 


10 


88.743 


45.902 


60.362 


9.663 


11 


90.494 


50.584 


60.486 


9.905 


12 


91.875 


55.374 


60.874 


9.939 


13 


92.059 


60.311 


61.644 


9.591 


14 


91.768 


65.298 


61.853 


9.945 


15 


90.631 


70.150 


62.256 


9.822 


16 


89.297 


74.966 


62.424 


9.986 


17 


86.789 


79.278 


62.770 


9.653 


18 


84.275 


83.598 


62.902 


9.997 


19 


81.106 


87.449 


63.255 


9.849 


20 


78.904 


91.933 


63.470 


9.728 


21 


76.920 


96.326 


64.797 


9.633 


22 


74.316 


100.527 


65.557 


9.796 


23 


72.143 


105.028 


65.683 


9.949 


24 


70.432 


109.626 


64.715 


9.780 


25 


68.153 


113.963 


63.718 


9.723 
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Figure D. 13 Course M (operator's display with 60° FOV) 




Figure D. 14 Course N (operator's display with 60° FOV) 
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Table D. 1 5 Course O 



Target 


X 


y 


z 


Best 8t 


1 


75 


5 


50 


N/A 


2 


75.910 


9.822 


50.957 


9.651 


3 


77.211 


14.460 


52.299 


9.602 


4 


78.500 


18.826 


54.367 


9.105 


5 


78.682 


23.509 


56.109 


9.130 


6 


78.331 


28.250 


57.657 


9.453 


7 


79.284 


32.909 


59.205 


9.185 


8 


80.718 


37.669 


59.737 


9.897 


9 


82.527 


42.322 


60.019 


9.952 


10 


84.463 


46.929 


60.170 


9.991 


11 


86.928 


51.277 


60.311 


9.927 


12 


89.000 


55.811 


59.923 


9.936 


13 


91.435 


60.063 


58.930 


9.756 


14 


93.206 


64.739 


58.944 


9.864 


15 


96.054 


68.755 


58.072 


9.523 


16 


98.649 


73.023 


57.847 


9.966 


17 


100.986 


77.376 


57.080 


9.867 


18 


104.258 


80.745 


55.364 


9.074 


19 


108.280 


83.431 


54.096 


9.457 


20 


111.762 


87.010 


53.838 


9.831 


21 


115.113 


90.688 


54.334 


9.950 


22 


118.488 


94.286 


53.521 


9.864 


23 


120.980 


98.614 


53.754 


9.792 


24 


122.835 


103.247 


54.071 


9.886 


25 


124.504 


107.871 


53.158 


9.806 
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Table D.16 Course P 



Target 


X 


y 


z 


Best 8t 


1 


75 


5 


50 


N/A 


2 


72.598 


9.372 


49.658 


9.584 


3 


69.070 


12.001 


47.283 


9.501 


4 


64.278 


13.316 


46.731 


8.949 


5 


59.900 


13.069 


44.328 


9.609 


6 


54.955 


13.750 


44.040 


9.983 


7 


50.197 


12.432 


43.248 


9.344 


8 


45.239 


12.926 


42.829 


9.532 


9 


40.819 


12.034 


44.988 


8.637 


10 


38.922 


10.074 


49.179 


16.763 


11 


39.445 


7.725 


53.562 


17.531 


12 


39.178 


7.113 


58.517 


19.820 


13 


37.208 


8.030 


63.020 


18.011 


14 


34.728 


10.445 


66.628 


14.434 


15 


33.378 


14.175 


69.671 


12.174 


16 


31.837 


16.656 


73.730 


16.233 


17 


32.710 


18.362 


78.348 


18.473 


18 


31.547 


19.258 


83.127 


19.118 


19 


29.540 


18.207 


87.585 


17.829 


20 


26.954 


15.938 


91.212 


14.510 


21 


23.758 


14.134 


94.609 


13.584 


22 


19.241 


13.240 


96.559 


8.860 


23 


14.933 


14.923 


98.456 


9.114 


24 


10.241 


16.585 


98.932 


9.935 


25 


5.979 


18.058 


101.093 


8.962 
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Figure D. 1 5 Course O (operator's display with 60° FOV) 




Figure D. 1 6 Course P (operator's display with 60° FOV) 
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Table D. 1 7 Course Q 



Target 


X 


y 


z 


Best 8t 


1 


75 


5 


50 


N/A 


2 


76.965 


9.488 


50.999 


9.010 


3 


78.914 


14.085 


51.257 


9.987 


4 


80.057 


18.154 


53.928 


10.684 


5 


81.442 


21.353 


57.512 


14.338 


6 


84.785 


23.831 


60.286 


11.093 


7 


89.534 


25.325 


60.742 


9.566 


8 


94.093 


26.833 


62.137 


9.558 


9 


97.370 


30.596 


62.454 


9.227 


10 


97.983 


35.556 


62.305 


9.426 


11 


98.032 


40.348 


63.733 


9.281 


12 


99.857 


44.961 


64.353 


9.166 


13 


103.293 


48.593 


64.334 


9.257 


14 


106.191 


52.163 


66.298 


8.822 


15 


109.898 


55.081 


67.954 


9.230 


16 


113.339 


57.474 


70.681 


10.907 


17 


116.610 


57.996 


74.426 


14.981 


18 


118.031 


58.438 


79.200 


19.094 


19 


120.155 


57.500 


83.628 


17.712 


20 


120.661 


57.006 


88.577 


19.799 


21 


123.270 


57.726 


92.782 


16.818 


22 


127.514 


57.380 


95.403 


10.483 


23 


131.744 


58.209 


97.936 


10.135 


24 


135.916 


60.827 


98.797 


9.851 


25 


140.313 


62.529 


100.460 


9.427 
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Table D. 18 Course R 



Target 


X 


y 


z 


Best St 


1 


75 


5 


50 


N/A 


2 


76.295 


9.827 


49.845 


9.654 


3 


79.038 


13.348 


47.593 


9.011 


4 


80.993 


17.733 


46.196 


9.422 


5 


84.768 


20.735 


44.877 


8.664 


6 


88.406 


24.027 


43.913 


9.805 


7 


90.185 


28.690 


43.604 


9.113 


8 


90.273 


33.487 


45.011 


9.341 


9 


91.413 


38.275 


45.893 


9.711 


10 


94.307 


41.610 


48.238 


9.379 


11 


96.604 


44.963 


51.151 


11.651 


12 


99.999 


45.848 


54.713 


14.250 


13 


102.820 


43.768 


'58.279 


14.261 


14 


103.495 


40.286 


61.803 


14.096 


15 


104.003 


37.783 


66.101 


17.195 


16 


104.024 


34.940 


70.214 


16.450 


17 


104.795 


32.056 


74.225 


16.044 


18 


105.128 


28.706 


77.922 


14.787 


19 


107.212 


24.785 


80.222 


9.200 


20 


110.117 


20.716 


80.238 


8.983 


21 


114.246 


18.089 


81.262 


9.452 


22 


118.870 


17.307 


79.527 


8.462 


23 


123.723 


16.223 


80.051 


9.048 


24 


128.234 


15.832 


77.931 


8.483 


25 


132.818 


17.512 


76.854 


9.276 
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Figure D. 17 Course Q (operator’s display with 60° FOV) 




Figure D. 18 Course R (operator's display with 60° FOV) 
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Table D.19 



Course X 



Target 


X 


y 


z 


1 


75 


5 


50 


2 


75.415 


9.975 


50.274 


3 


75.804 


14.960 


50.278 


4 


76.682 


19.878 


50.081 


5 


77.443 


24.817 


49.909 


6 


78.169 


29.761 


50.092 


7 


79.052 


34.674 


49.800 


8 


80.541 


39.441 


50.037 


9 


81.479 


44.352 


50.041 


10 


82.073 


49.286 


50.595 


11 


82.721 


54.141 


51.598 


12 


83.762 


58.762 


53.199 


13 


85.367 


62.995 


55.321 


14 


87.501 


66.919 


57.568 


15 


89.307 


71.089 


59.653 


16 


91.327 


75.215 


61.627 


17 


93.395 


79.426 


63.357 


18 


95.436 


83.409 


65.587 


19 


98.061 


87.359 


67.170 


20 


100.328 


91.501 


68.815 


21 


102.772 


95.558 


70.417 


22 


105.399 


99.343 


72.359 


23 


107.766 


103.514 


73.773 


24 


109.633 


107.777 


75.602 


25 


111.801 


111.855 


77.516 
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Figure D. 19 Course X (operator's display with 60° FOV) 




Figure D.20 Course X (operator's display with 30° FOV) 
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Figure D.21 Course X (operator's display with 15° FOV) 
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APPENDIX E 



Raw Data 



Tables E.l through E.12 list the data for each of the four operators over all seven 
tests. The tabulated values are the time differences between successive targets in seconds. 
Value 1 is the time difference between hitting targets 1 and 2, value 2 for targets 2 and 3, 
etc. The total for each column is indicated at the bottom of the column. For all tests the 
course letter is specified at the top of its respective column. For Test 1 the operator's field 
of view is also shown. 
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Table E.l 



Operator I, Test 1 





60° FOV 


30° FOV 


15° FOV 


A 


D 


G 


B 


E 


H 


C 


F 


I 


1 


16 


16 


13 


55 


53 


35 


12 


53 


42 


2 


10 


17 


39 


50 


59 


38 


39 


90 


49 


3 


14 


15 


16 


33 


67 


101 


28 


74 


29 


4 


15 


37 


28 


40 


78 


59 


33 


31 


45 


5 


15 


22 


32 


36 


79 


48 


35 


82 


79 


6 


15 


22 


22 


33 


55 


34 


75 


50 


43 


7 


14 


15 


14 


33 


27 


23 


72 


96 


32 


8 


11 


17 


28 


194 


25 


39 


83 


60 


33 


9 


11 


27 


129 


48 


11 


11 


76 


21 


219 


10 


11 


35 


88 


64 


32 


36 


92 


33 


27 


11 


18 


29 


180 


31 


82 


22 


89 


43 


189 


12 


17 


24 


67 


116 


37 


41 


51 


31 


221 


13 


15 


18 


34 


75 


14 


106 


63 


19 


46 


14 


14 


20 


18 


39 


68 


44 


69 


61 


47 


15 


46 


19 


25 


73 


69 


94 


32 


26 


59 


16 


36 


49 


44 


77 


157 


52 


43 


93 


109 


17 


16 


21 


78 


50 


38 


195 


56 


33 


114 


18 


15 


31 


93 


24 


14 


48 


42 


22 


25 


19 


13 


18 


104 


51 


21 


46 


89 


25 


41 


20 


12 


16 


38 


85 


47 


60 


35 


21 


83 


21 


11 


21 


51 


30 


17 


57 


40 


24 


22 


22 


11 


16 


38 


47 


11 


26 


34 


19 


21 


23 


10 


18 


27 


152 


41 


42 


33 


16 


28 


24 


66 


12 


12 


52 


18 


60 


32 


13 


138 


Total 


432 


535 


1218 


1488 


1120 


1317 


1253 


1036 


1741 
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Table E.2 Operator I - Tests 2, 3, and 4 





Test 2 


Test 3 


Test 4 


J 


M 


P 


K 


N 


Q 


A 


E 


I 


1 


16 


16 


41 


15 


13 


20 


18 


19 


16 


2 


16 


11 


36 


14 


17 


17 


13 


24 


26 


3 


11 


15 


21 


10 


15 


20 


11 


16 


18 


4 


17 


29 


28 


14 


22 


28 


14 


23 


28 


5 


24 


46 


19 


18 


18 


36 


15 


13 


36 


6 


19 


12 


26 


16 


55 


91 


32 


16 


13 


7 


29 


17 


14 


38 


22 


27 


16 


25 


19 


8 


13 


16 


53 


18 


13 


31 


11 


12 


47 


9 


14 


22 


37 


14 


15 


17 


10 


20 


16 


10 


24 


37 


93 


15 


12 


50 


11 


17 


17 


11 


28 


18 


105 


13 


13 


16 


15 


13 


53 


12 


24 


21 


65 


11 


10 


23 


18 


11 


144 


13 


12 


25 


88 


13 


12 


40 


24 


20 


19 


14 


13 


23 


27 


13 


11 


21 


11 


19 


29 


15 


18 


26 


52 


47 


16 


22 


12 


11 


33 


16 


25 


38 


47 


16 


12 


48 


12 


15 


49 


17 


18 


34 


47 


22 


18 


30 


13 


13 


28 


18 


27 


17 


69 


16 


18 


69 


13 


12 


20 


19 


22 


26 


102 


18 


14 


70 


14 


14 


22 


20 


42 


20 


32 


25 


15 


36 


11 


15 


30 


21 


19 


20 


19 


20 


11 


37 


14 


10 


17 


22 


25 


22 


32 


39 


9 


32 


16 


12 


10 


23 


18 


12 


25 


22 


16 


12 


12 


21 


14 


24 


18 


10 


30 


23 


10 


14 


11 


16 


23 


Total 


492 


533 


1108 


470 


387 


807 


347 


387 


727 
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Table E.3 Operator I - Tests 5, 6, and 7 





Test 5 


Test 6 


Test 7 


B 


F 


G 


L 


0 


R 


C 


D 


H 


1 


11 


12 


50 


10 


17 


29 


10 


25 


56 


2 


48 


19 


39 


10 


11 


31 


10 


22 


20 


3 


18 


17 


14 


11 


32 


24 


10 


16 


22 


4 


15 


13 


25 


9 


15 


23 


18 


29 


38 


5 


14 


13 


25 


12 


12 


18 


19 


31 


47 


6 


17 


16 


20 


12 


21 


67 


22 


28 


13 


7 


17 


16 


17 


12 


14 


25 


19 


35 


18 


8 


10 


10 


20 


13 


14 


30 


21 


29 


47 


9 


10 


10 


67 


22 


14 


24 


24 


27 


15 


10 


28 


19 


62 


18 


17 


22 


48 


26 


18 


11 


39 


18 


38 


18 


18 


32 


19 


37 


37 


12 


29 


14 


49 


20 


16 


85 


20 


27 


17 


13 


29 


18 


91 


15 


16 


50 


18 


36 


37 


14 


23 


15 


39 


12 


33 


25 


12 


27 


34 


15 


22 


10 


33 


13 


21 


30 


10 


20 


22 


16 


23 


31 


31 


12 


13 


25 


10 


24 


55 


17 


27 


13 


41 


13 


33 


24 


14 


37 


37 


18 


22 


26 


86 


23 


21 


32 


51 


18 


60 


19 


26 


13 


45 


14 


14 


17 


28 


32 


54 


20 


21 


22 


28 


11 


14 


44 


16 


24 


25 


21 


26 


21 


16 


11 


18 


19 


15 


33 


27 


22 


55 


14 


20 


10 


23 


17 


16 


15 


42 


23 


28 


12 


18 


11 


13 


17 


17 


10 


20 


24 


24 


10 


14 


15 


10 


16 


15 


11 


32 


Total 


582 


382 


888 


327 


430 


726 


462 


619 


793 
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Table E.4 Operator II - Test 1 





60° FOV 


30° FOV 


15° FOV 


J 


M 


P 


K 


N 


Q 


L 


O 


R 


1 


14 


15 


18 


22 


39 


40 


27 


32 


25 


2 


15 


32 


88 


15 


31 


49 


22 


44 


68 


3 


12 


44 


13 


43 


76 


38 


51 


28 


61 


4 


10 


18 


53 


20 


45 


38 


16 


31 


26 


5 


12 


57 


12 


25 


32 


203 


33 


38 


41 


6 


10 


35 


97 


15 


53 


52 


67 


124 


19 


7 


36 


23 


20 


76 


29 


35 


71 


29 


47 


8 


66 


21 


75 


15 


30 


52 


78 


92 


88 


9 


19 


15 


239 


24 


36 


74 


32 


37 


43 


10 


23 


107 


77 


16 


33 


23 


32 


115 


77 


11 


19 


47 


51 


116 


62 


16 


34 


48 


87 


12 


24 


44 


69 


37 


24 


93 


36 


80 


115 


13 


31 


20 


55 


30 


26 


185 


25 


59 


98 


14 


63 


30 


54 


59 


77 


27 


41 


99 


94 


15 


19 


21 


56 


30 


13 


43 


36 


44 


86 


16 


25 


27 


38 


110 


49 


32 


20 


59 


47 


17 


14 


20 


259 


87 


16 


63 


69 


245 


347 


18 


82 


20 


132 


31 


20 


59 


17 


56 


130 


19 


42 


16 


87 


61 


24 


43 


34 


130 


220 


20 


27 


13 


66 


44 


35 


63 


34 


263 


133 


21 


11 


12 


85 


60 


43 


57 


274 


43 


113 


22 


19 


27 


34 


52 


63 


30 


31 


57 


97 


23 


29 


19 


51 


42 


17 


29 


85 


71 


160 


24 


87 


10 


34 


70 


86 


19 


125 


72 


61 


Total 


709 


693 


1763 


1100 


959 


1363 


1290 


1896 


2283 
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Table E.5 Operator II - Tests 2, 3, and 4 





Test 2 


Test 3 


Test 4 


A 


D 


G 


B 


E 


H 


J 


N 


R 


1 


20 


16 


21 


12 


25 


30 


12 


20 


30 


2 


12 


12 


28 


22 


25 


17 


12 


13 


30 


3 


14 


19 


23 


14 


50 


17 


10 


14 


32 


4 


13 


55 


26 


14 


36 


30 


15 


22 


40 


5 


25 


34 


21 


16 


23 


43 


14 


16 


18 


6 


20 


22 


26 


22 


16 


82 


12 


23 


18 


7 


21 


25 


17 


22 


22 


39 


11 


23 


11 


8 


14 


18 


17 


33 


23 


59 


10 


17 


15 


9 


16 


25 


213 


32 


24 


17 


11 


17 


37 


10 


19 


33 


45 


33 


13 


23 


10 


19 


34 


11 


22 


35 


82 


28 


18 


18 


10 


16 


72 


12 


26 


22 


150 


54 


15 


17 


13 


12 


56 


13 


20 


125 


53 


35 


15 


41 


18 


10 


30 


14 


25 


32 


76 


33 


80 


22 


15 


12 


71 


15 


20 


21 


45 


30 


31 


27 


14 


17 


79 


16 


17 


27 


28 


38 


31 


48 


17 


46 


30 


17 


13 


23 


78 


36 


30 


44 


17 


16 


35 


18 


15 


32 


49 


33 


30 


54 


17 


22 


23 


19 


10 


25 


99 


25 


40 


66 


18 


12 


16 


20 


10 


20 


29 


32 


28 


64 


18 


15 


24 


21 


11 


17 


51 


58 


45 


74 


20 


14 


33 


22 


18 


22 


60 


30 


49 


65 


25 


14 


20 


23 


17 


16 


54 


32 


26 


15 


27 


9 


23 


24 


35 


24 


39 


28 


36 


48 


17 


15 


48 


Total 


433 


700 


1330 


712 


731 


960 


363 


414 


825 
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Table E.6 



Operator II - Tests 5, 6, and 7 





Test 5 


Test 6 


Test 7 


K 


O 


P 


C 


F 


I 


L 


M 


Q 


1 


11 


22 


50 


13 


12 


31 


20 


25 


19 


2 


10 


20 


37 


27 


14 


48 


17 


20 


28 


3 


12 


25 


25 


30 


21 


26 


17 


24 


41 


4 


13 


23 


25 


21 


10 


32 


53 


20 


40 


5 


11 


22 


25 


17 


26 


28 


28 


21 


50 


6 


12 


22 


13 


27 


20 


22 


20 


58 


38 


7 


10 


21 


14 


17 


14 


17 


23 


26 


28 


8 


11 


25 


18 


16 


13 


20 


45 


28 


21 


9 


16 


15 


37 


25 


19 


22 


33 


22 


27 


10 


18 


13 


52 


22 


19 


23 


37 


23 


22 


11 


18 


14 


54 


19 


18 


74 


19 


23 


37 


12 


21 


43 


32 


19 


22 


136 


32 


23 


14 


13 


25 


25 


39 


16 


12 


72 


22 


25 


27 


14 


26 


18 


43 


23 


18 


87 


14 


23 


28 


15 


29 


15 


36 


27 


17 


23 


26 


26 


53 


16 


30 


20 


42 


16 


13 


40 


53 


38 


76 


17 


20 


23 


35 


16 


25 


41 


25 


37 


181 


18 


19 


28 


54 


16 


21 


38 


24 


32 


50 


19 


25 


33 


32 


13 


12 


22 


23 


20 


46 


20 


21 


19 


28 


15 


19 


28 


22 


19 


41 


21 


30 


14 


49 


17 


21 


45 


26 


17 


37 


22 


80 


11 


42 


16 


11 


94 


16 


20 


44 


23 


37 


12 


19 


19 


16 


32 


19 


27 


42 


24 


31 


12 


28 


19 


38 


51 


38 


12 


16 


Total 


536 


495 


829 


466 


431 


1052 


652 


609 


1006 
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Table E.7 Operator III - Test 1 





60° FOV 


30° FOV 


15° FOV 


B 


E 


H 


C 


F 


I 


A 


D 


G 


1 


10 


29 


21 


10 


54 


31 


14 


10 


222 


2 


12 


78 


16 


11 


40 


49 


11 


10 


398 


3 


11 


39 


49 


13 


53 


65 


20 


185 


28 


4 


10 


29 


39 


64 


45 


88 


55 


153 


287 


5 


11 


40 


32 


52 


61 


47 


24 


33 


46 


6 


11 


34 


18 


69 


47 


39 


18 


30 


29 


7 


10 


28 


23 


54 


35 


113 


29 


46 


32 


8 


10 


54 


27 


76 


44 


170 


13 


19 


35 


9 


10 


14 


18 


59 


47 


149 


9 


26 


124 


10 


36 


11 


27 


67 


69 


109 


11 


31 


182 


11 


9 


11 


17 


101 


45 


203 


14 


116 


68 


12 


15 


12 


10 


70 


50 


95 


17 


28 


82 


13 


14 


110 


19 


76 


86 


61 


53 


24 


101 


14 


16 


33 


34 


63 


99 


90 


24 


20 


71 


15 


68 


17 


20 


48 


47 


93 


28 


21 


42 


16 


57 


12 


76 


57 


43 


32 


41 


37 


72 


17 


17 


18 


25 


55 


59 


30 


17 


37 


37 


18 


15 


42 


86 


65 


61 


120 


11 


158 


102 


19 


62 


17 


256 


70 


40 


17 


10 


32 


230 


20 


42 


29 


101 


69 


46 


17 


9 


34 


397 


21 


19 


30 


45 


73 


53 


21 


10 


21 


127 


22 


142 


28 


41 


85 


29 


17 


13 


62 


26 


23 


99 


20 


46 


67 


62 


26 


14 


9 


23 


24 


66 


38 


117 


43 


31 


65 


13 


10 


44 


Total 


772 


773 


1163 


1417 


1246 


1747 


478 


1152 


2805 
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Table E.8 Operator III - Tests 2, 3, and 4 





Test 2 


Test 3 


Test 4 


K 


N 


Q 


L 


0 


R 


B 


F 


G 


1 


14 


28 


26 


10 


16 


13 


19 


15 


63 


2 


30 


18 


18 


10 


18 


20 


36 


12 


35 


3 


33 


34 


28 


33 


21 


11 


26 


16 


79 


4 


34 


18 


51 


28 


32 


22 


12 


38 


25 


5 


20 


19 


29 


102 


13 


15 


21 


38 


26 


6 


29 


21 


23 


44 


40 


11 


58 


24 


29 


7 


20 


17 


72 


12 


17 


14 


12 


13 


18 


8 


19 


31 


43 


17 


20 


21 


21 


17 


17 


9 


13 


28 


17 


21 


79 


14 


16 


24 


73 


10 


19 


15 


76 


22 


64 


23 


26 


10 


110 


11 


36 


17 


36 


20 


22 


54 


20 


13 


94 


12 


28 


22 


15 


15 


12 


32 


20 


14 


45 


13 


68 


14 


65 


58 


34 


26 


25 


13 


55 


14 


32 


20 


29 


10 


16 


70 


17 


15 


33 


15 


20 


23 


30 


10 


16 


40 


19 


12 


37 


16 


67 


31 


47 


11 


43 


44 


26 


16 


55 


17 


27 


26 


38 


12 


22 


43 


18 


23 


37 


18 


23 


21 


51 


33 


64 


11 


19 


22 


74 


19 


24 


25 


39 


15 


30 


21 


34 


16 


54 


20 


29 


23 


47 


97 


12 


54 


32 


13 


29 


21 


33 


15 


41 


21 


30 


28 


36 


17 


19 


22 


46 


11 


31 


16 


20 


20 


30 


17 


23 


23 


40 


18 


62 


9 


11 


15 


40 


11 


14 


24 


36 


12 


21 


9 


19 


55 


36 


13 


46 


Total 


740 


507 


935 


635 


671 


677 


619 


422 


1090 
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Table E.9 Operator III - Tests 5, 6, and 7 





Test 5 


Test 6 


Test 7 


C 


D 


H 


J 


M 


P 


A 


E 


I 


1 


10 


21 


15 


11 


26 


11 


16 


24 


20 


2 


10 


10 


10 


9 


15 


34 


19 


23 


62 


3 


10 


20 


23 


43 


22 


19 


21 


51 


59 


4 


13 


58 


33 


16 


37 


23 


20 


23 


35 


5 


21 


14 


23 


12 


69 


15 


12 


35 


46 


6 


46 


14 


20 


11 


24 


45 


21 


27 


20 


7 


17 


21 


15 


10 


18 


19 


24 


58 


41 


8 


44 


17 


30 


10 


17 


36 


15 


44 


38 


9 


41 


21 


16 


11 


13 


49 


24 


25 


25 


10 


54 


14 


36 


10 


69 


72 


26 


19 


32 


11 


19 


26 


28 


14 


28 


83 


21 


20 


108 


12 


48 


21 


17 


10 


35 


71 


28 


30 


98 


13 


24 


22 


22 


9 


31 


68 


56 


19 


66 


14 


20 


22 


14 


12 


30 


75 


22 


18 


27 


15 


17 


23 


21 


10 


26 


63 


19 


17 


29 


16 


15 


24 


33 


18 


31 


50 


30 


19 


58 


17 


14 


21 


51 


16 


24 


58 


35 


13 


51 


18 


15 


21 


42 


11 


30 


48 


11 


32 


67 


19 


24 


20 


52 


57 


32 


49 


24 


20 


35 


20 


10 


38 


39 


15 


35 


51 


26 


37 


20 


21 


12 


11 


48 


14 


25 


38 


17 


104 


35 


22 


10 


10 


52 


16 


23 


65 


26 


36 


17 


23 


25 


10 


22 


9 


19 


26 


50 


28 


19 


24 


23 


10 


18 


16 


17 


14 


24 


28 


26 


Total 


542 


489 


680 


370 


696 


1082 


587 


750 


1034 
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Table E. 10 Operator IV - Test 1 





60° FOV 


30° FOV 


15° FOV 


K 


N 


Q 


L 


O 


R 


J 


M 


P 


1 


38 


23 


25 


40 


51 


79 


76 


46 


35 


2 


37 


46 


32 


44 


46 


77 


86 


30 


51 


3 


30 


27 


51 


38 


69 


51 


67 


132 


48 


4 


49 


25 


38 


62 


39 


61 


48 


61 


110 


5 


54 


41 


38 


70 


39 


83 


64 


297 


54 


6 


39 


49 


20 


39 


24 


19 


53 


50 


277 


7 


67 


25 


23 


43 


27 


61 


43 


73 


56 


8 


51 


48 


33 


79 


18 


48 


30 


24 


316 


9 


28 


42 


24 


26 


15 


183 


21 


51 


140 


10 


25 


36 


22 


36 


14 


61 


49 


52 


310 


11 


22 


49 


20 


40 


11 


253 


46 


34 


219 


12 


23 


25 


20 


25 


29 


270 


31 


51 


152 


13 


33 


15 


34 


26 


37 


171 


104 


202 


179 


14 


42 


31 


31 


41 


42 


282 


135 


216 


258 


15 


45 


19 


35 


22 


38 


119 


43 


45 


90 


16 


32 


21 


56 


17 


14 


141 


97 


61 


127 


17 


30 


24 


46 


20 


46 


95 


40 


54 


96 


18 


49 


18 


74 


33 


48 


61 


99 


176 


88 


19 


39 


21 


33 


29 


15 


20 


127 


58 


104 


20 


66 


22 


145 


26 


25 


69 


121 


69 


64 


21 


75 


35 


28 


20 


171 


79 


46 


14 


44 


22 


44 


47 


40 


19 


53 


66 


32 


40 


63 


23 


36 


31 


66 


38 


35 


100 


57 


41 


121 


24 


45 


14 


46 


47 


39 


20 


60 


30 


79 


Total 


999 


734 


980 


880 


945 


2469 


1575 


1907 


3081 
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Table E.l 1 Operator IV - Tests 2, 3, and 4 





Test 2 


Test 3 


Test 4 


B 


E 


H 


C 


F 


I 


K 


0 


P 


1 


11 


17 


19 


11 


15 


21 


12 


11 


29 


2 


20 


22 


13 


12 


38 


23 


23 


17 


41 


3 


16 


21 


40 


10 


21 


18 


18 


24 


18 


4 


17 


27 


41 


12 


16 


53 


35 


26 


23 


5 


17 


37 


54 


17 


12 


30 


21 


22 


16 


6 


14 


21 


21 


15 


16 


22 


16 


17 


15 


7 


13 


17 


28 


13 


19 


17 


17 


13 


18 


8 


22 


22 


19 


22 


19 


14 


11 


30 


22 


9 


20 


19 


19 


18 


18 


22 


11 


18 


46 


10 


35 


22 


16 


55 


16 


24 


13 


10 


56 


11 


36 


26 


16 


19 


18 


109 


15 


43 


25 


12 


18 


13 


58 


17 


31 


77 


16 


16 


152 


13 


14 


10 


20 


14 


16 


77 


14 


14 


48 


14 


17 


16 


23 


11 


21 


109 


14 


17 


22 


15 


15 


12 


32 


13 


36 


29 


18 


18 


35 


16 


17 


16 


60 


14 


27 


36 


23 


13 


103 


17 


53 


12 


59 


12 


16 


26 


19 


16 


76 


18 


18 


20 


66 


14 


11 


23 


18 


13 


84 


19 


52 


18 


133 


12 


14 


11 


21 


11 


59 


20 


29 


18 


36 


15 


11 


19 


22 


13 


31 


21 


28 


20 


42 


11 


11 


20 


67 


14 


16 


22 


68 


17 


31 


12 


18 


17 


29 


14 


37 


23 


22 


11 


22 


10 


15 


13 


53 


10 


11 


24 


17 


11 


17 


9 


10 


17 


43 


10 


28 


Total 


589 


445 


885 


368 


445 


827 


549 


410 


1011 



114 



Table E.12 Operator IV - Tests 5, 6, and 7 





Test 5 


Test 6 


Test 7 


L 


M 


Q 


A 


D 


G 


J 


N 


R 


1 


11 


17 


17 


19 


15 


25 


12 


32 


12 


2 


12 


11 


11 


29 


17 


23 


14 


34 


35 


3 


10 


17 


20 


20 


13 


19 


23 


28 


71 


4 


14 


13 


24 


13 


60 


28 


17 


24 


24 


5 


13 


11 


21 


12 


30 


16 


13 


23 


18 


6 


12 


11 


16 


13 


12 


15 


13 


22 


40 


7 


19 


13 


58 


16 


13 


18 


14 


24 


25 


8 


13 


11 


16 


13 


14 


24 


14 


25 


29 


9 


12 


12 


15 


22 


19 


45 


15 


14 


28 


10 


17 


17 


38 


20 


29 


84 


12 


20 


23 


11 


13 


20 


14 


10 


20 


43 


21 


16 


60 


12 


14 


17 


15 


11 


19 


61 


26 


19 


61 


13 


39 


13 


17 


10 


25 


58 


28 


12 


53 


14 


11 


22 


19 


12 


22 


99 


25 


20 


172 


15 


10 


18 


124 


11 


13 


51 


20 


14 


42 


16 


10 


19 


32 


10 


75 


40 


35 


23 


44 


17 


52 


17 


75 


11 


29 


92 


24 


17 


31 


18 


14 


27 


91 


9 


21 


72 


13 


18 


22 


19 


12 


14 


45 


11 


26 


73 


13 


19 


11 


20 


12 


16 


52 


20 


20 


23 


25 


17 


26 


21 


15 


9 


35 


17 


20 


16 


32 


23 


26 


22 


12 


10 


25 


13 


17 


22 


18 


13 


19 


23 


10 


10 


14 


10 


12 


15 


14 


20 


24 


24 


9 


10 


23 


48 


11 


28 


26 


14 


18 


Total 


366 


355 


817 


380 


552 


990 


467 


491 


914 
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APPENDIX F 



Normalized Data 



Tables F.l through F.12 list the data for each of the four operators over all seven 
tests. The tabulated values are normalized values from the associated tables in Appendix E. 
The data were normalized to remove effects properly attributed to variations in the level of 
course difficulty related to the Jason vehicle simulation model. The total, mean value, and 
standard deviation (a) for each column are indicated at the bottom of the column. For all 
tests the course letter is specified at the top of its respective column. For Test 1 the 
operator’s field of view is also shown. 

The normalization procedure divided each time difference by the best time between 
the associated targets from Appendix D to arrive at the normalized value (dimensionless). 
Because the logged times represented the most recently passed whole second, while the 
best times between targets were calculated to a greater degree of precision, it was possible 
to achieve a normalized value of 0.9. 
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Table F . 1 Operator I - Test 1 





60 ° FOV 


30 ° FOV 


15 ° FOV 


A 


D 


G 


B 


E 


H 


C 


F 


I 


1 


1.6 


1.6 


1.4 


5.6 


5.4 


3.9 


1.2 


5.5 


4.4 


2 


1.0 


1.7 


4.1 


5.1 


6.1 


3.8 


3.9 


9.4 


5.4 


3 


1.4 


1.6 


1.8 


3.4 


7.2 


9.5 


2.8 


8.1 


3.1 


4 


1.5 


4.0 


2.9 


4.1 


8.3 


4.1 


3.3 


3.4 


5.2 


5 


1.5 


2.3 


3.2 


3.7 


8.0 


4.3 


3.6 


8.7 


8.1 


6 


1.5 


2.3 


2.4 


3.3 


5.9 


3.6 


7.7 


5.4 


4.7 


7 


1.4 


1.6 


1.5 


3.3 


2.9 


2.4 


7.3 


9.7 


3.4 


8 


1.1 


1.7 


3.2 


20.0 


2.6 


4.2 


8.6 


6.0 


3.4 


9 


1.1 


3.0 


7.7 


5.1 


1.1 


1.2 


7.9 


2.1 


23.4 


10 


1.1 


3.0 


5.0 


7.0 


3.2 


3.9 


9.9 


3.3 


2.3 


11 


1.8 


2.7 


9.1 


3.4 


8.3 


2.4 


9.5 


4.3 


13.3 


12 


1.7 


2.6 


3.7 


12.8 


3.9 


4.4 


5.3 


3.2 


15.5 


13 


1.5 


2.0 


2.4 


7.6 


1.4 


12.0 


6.4 


1.9 


3.3 


14 


1.4 


2.2 


1.5 


3.3 


6.9 


4.8 


6.9 


6.4 


2.7 


15 


4.7 


2.1 


1.5 


5.9 


6.9 


8.6 


3.2 


2.6 


3.6 


16 


3.8 


4.0 


2.4 


6.1 


16.3 


3.5 


4.3 


9.4 


6.8 


17 


1.6 


1.9 


4.1 


4.7 


3.8 


10.2 


5.6 


3.6 


7.7 


18 


1.5 


3.5 


5.2 


2.1 


1.4 


2.7 


4.3 


2.3 


2.7 


19 


1.3 


2.0 


7.2 


4.0 


2.2 


2.3 


9.3 


2.5 


4.6 


20 


1.2 


1.7 


2.8 


6.1 


4.9 


3.6 


3.8 


2.1 


8.8 


21 


1.1 


2.2 


5.8 


2.1 


1.7 


5.4 


4.1 


2.4 


2.6 


22 


1.1 


1.6 


4.2 


3.0 


1.1 


2.6 


3.6 


1.9 


2.3 


23 


1.0 


1.8 


2.7 


9.7 


4.2 


4.3 


3.5 


1.6 


3.3 


24 


6.9 


1.2 


1.3 


3.1 


1.9 


6.4 


3.5 


1.3 


14.9 


Total 


44.0 


54.3 


86.9 


134.4 


115.6 


114.0 


129.3 


107.5 


155.3 


Mean 


1.8 


2.3 


3.6 


5.6 


4.8 


4.8 


5.4 


4.5 


6.5 


a 


1.4 


0.7 


2.1 


3.8 


3.4 


2.7 


2.4 


2.7 


5.2 
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Table F.2 Operator I - Tests 2, 3, and 4 





Test 2 


Test 3 


Test 4 


J 


M 


P 


K 


N 


Q 


A 


E 


I 


1 


1.6 


1.6 


4.3 


1.5 


1.3 


2.2 


1.8 


1.9 


1.7 


2 


1.6 


1.1 


3.8 


1.4 


1.8 


1.7 


1.3 


2.5 


2.9 


3 


1.1 


1.6 


2.3 


1.0 


1.6 


1.9 


1.1 


1.7 


1.9 


4 


1.7 


3.1 


2.9 


1.4 


2.3 


2.0 


1.4 


2.5 


3.2 


5 


2.4 


4.8 


1.9 


1.8 


1.8 


3.2 


1.5 


1.3 


3.7 


6 


1.9 


1.2 


2.8 


1.6 


5.9 


9.5 


3.2 


1.7 


1.4 


7 


2.9 


1.8 


1.5 


3.8 


2.4 


2.8 


1.6 


2.7 


2.0 


8 


1.3 


1.6 


6.1 


1.9 


1.4 


3.4 


1.1 


1.3 


4.8 


9 


1.4 


2.4 


2.2 


1.5 


1.6 


1.8 


1.0 


2.1 


1.7 


10 


2.4 


3.2 


5.3 


1.6 


1.2 


5.4 


1.1 


1.7 


1.5 


11 


2.8 


1.7 


5.3 


1.4 


1.3 


1.7 


1.5 


1.3 


3.7 


12 


2.4 


2.3 


3.6 


1.2 


1.0 


2.5 


1.8 


1.1 


10.1 


13 


1.2 


2.7 


6.1 


1.3 


1.2 


4.5 


2.4 


2.0 


1.3 


14 


1.3 


2.5 


2.2 


1.1 


1.1 


2.3 


1.1 


1.9 


1.7 


15 


1.8 


2.9 


3.2 


3.8 


1.6 


2.0 


1.2 


1.1 


2.0 


16 


2.6 


3.1 


2.5 


1.3 


1.2 


3.2 


1.3 


1.6 


3.1 


17 


1.8 


3.1 


2.5 


2.1 


1.8 


1.6 


1.3 


1.3 


1.9 


18 


2.7 


1.9 


3.9 


1.4 


1.8 


3.9 


1.3 


1.2 


2.2 


19 


2.2 


2.9 


7.0 


1.4 


1.4 


3.5 


1.4 


1.4 


2.4 


20 


4.2 


2.2 


2.4 


1.8 


1.6 


2.1 


1.1 


1.6 


3.2 


21 


1.9 


2.1 


2.1 


1.4 


1.1 


3.5 


1.4 


1.0 


2.0 


22 


2.5 


2.3 


3.5 


2.5 


0.9 


3.2 


1.6 


1.2 


1.1 


23 


1.9 


1.2 


2.5 


1.4 


1.6 


1.2 


1.2 


2.1 


1.7 


24 


1.9 


1.0 


3.3 


1.4 


1.0 


1.5 


1.1 


1.6 


2.5 


Total 


49.8 


54.3 


83.3 


41.1 


40.1 


70.7 


35.1 


40.0 


63.7 


Mean 


2.1 


2.3 


3.5 


1.7 


1.7 


2.9 


1.5 


1.7 


2.7 


a 


0.7 


0.9 


1.5 


0.7 


1.0 


1.7 


0.5 


0.5 


1.8 
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Table F.3 Operator I - Tests 5, 6, and 7 





Test 5 


Test 6 


Test 7 


B 


F 


G 


L 


0 


R 


C 


D 


H 


1 


1.1 


1.2 


5.2 


1.0 


1.8 


3.0 


1.0 


2.5 


6.2 


2 


4.9 


2.0 


4.1 


1.0 


1.1 


3.4 


1.0 


2.2 


2.0 


3 


1.8 


1.9 


1.6 


1.1 


3.5 


2.5 


1.0 


1.7 


2.1 


4 


1.5 


1.4 


2.6 


0.9 


1.6 


2.7 


1.8 


3.1 


2.7 


5 


1.4 


1.4 


2.5 


1.2 


1.3 


1.8 


2.0 


3.2 


4.2 


6 


1.7 


1.7 


2.1 


1.2 


2.3 


7.4 


2.2 


2.9 


1.4 


7 


1.7 


1.6 


1.8 


1.2 


1.4 


2.7 


1.9 


3.6 


1.9 


8 


1.0 


1.0 


2.3 


1.3 


1.4 


3.1 


2.2 


3.0 


5.1 


9 


1.1 


1.0 


4.0 


2.3 


1.4 


2.6 


2.5 


3.0 


1.6 


10 


3.0 


1.9 


3.5 


1.9 


1.7 


1.9 


5.1 


2.2 


1.9 


11 


4.2 


1.8 


1.9 


1.9 


1.8 


2.2 


2.0 


3.5 


4.0 


12 


3.2 


1.4 


2.7 


2.1 


1.6 


6.0 


2.1 


2.9 


1.8 


13 


3.0 


1.8 


6.3 


1.5 


1.6 


3.5 


1.8 


3.9 


4.2 


14 


2.0 


1.6 


3.2 


1.2 


3.5 


1.5 


1.2 


3.0 


3.7 


15 


1.8 


1.0 


2.0 


1.3 


2.1 


1.8 


1.0 


2.2 


2.0 


16 


1.8 


3.1 


1.7 


1.2 


1.3 


1.6 


1.0 


2.0 


3.7 


17 


2.5 


1.4 


2.1 


1.3 


3.6 


1.6 


1.4 


3.3 


1.9 


18 


1.9 


2.7 


4.8 


2.4 


2.2 


3.5 


5.2 


2.0 


3.4 


19 


2.1 


1.3 


3.1 


1.5 


1.4 


1.9 


2.9 


3.6 


2.7 


20 


1.5 


2.2 


2.1 


1.2 


1.4 


4.7 


1.7 


2.6 


1.5 


21 


1.8 


2.1 


1.8 


1.1 


1.8 


2.2 


1.5 


3.4 


2.6 


22 


3.5 


1.4 


2.2 


1.1 


2.3 


1.9 


1.7 


1.5 


4.1 


23 


1.8 


1.2 


1.8 


1.2 


1.3 


2.0 


1.8 


1.0 


2.0 


24 


1.4 


1.0 


1.6 


1.6 


1.0 


1.7 


1.6 


1.1 


3.4 


Total 


51.9 


39.5 


67.1 


33.7 


44.7 


67.1 


47.8 


63.6 


70.2 


Mean 


2.2 


1.6 


2.8 


1.4 


1.9 


2.8 


2.0 


2.7 


2.9 


a 


1.0 


0.5 


1.2 


0.4 


0.7 


1.4 


1.1 


0.8 


1.2 
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Table F.4 Operator II - Test 1 





60 ° FOV 


30 ° FOV 


15 ° FOV 


J 


M 


P 


K 


N 


Q 


L 


O 


R 


1 


1.4 


1.5 


1.9 


2.2 


4.0 


4.4 


2.7 


3.3 


2.6 


2 


1.5 


3.3 


9.3 


1.5 


3.2 


4.9 


2.2 


4.6 


7.5 


3 


1.2 


4.6 


1.5 


4.4 


8.2 


3.6 


5.1 


3.1 


6.5 


4 


1.0 


1.9 


5.5 


2.0 


4.8 


2.7 


1.6 


3.4 


3.0 


5 


1.2 


5.9 


1.2 


2.5 


3.2 


18.3 


3.4 


4.0 


4.2 


6 


1.0 


3.6 


10.4 


1.5 


5.7 


5.4 


6.8 


13.5 


2.1 


7 


3.6 


2.4 


2.1 


7.7 


3.1 


3.7 


7.2 


2.9 


5.0 


8 


6.6 


2.1 


8.7 


1.5 


3.1 


5.6 


8.0 


9.2 


9.1 


9 


1.9 


1.7 


14.3 


2.5 


3.7 


7.9 


3.3 


3.7 


4.6 


10 


2.3 


9.2 


4.4 


1.7 


3.3 


2.5 


3.4 


11.6 


6.6 


11 


1.9 


4.4 


2.6 


12.6 


6.2 


1.7 


3.6 


4.8 


6.1 


12 


2.4 


4.7 


3.8 


4.1 


2.5 


10.0 


3.7 


8.2 


8.1 


13 


3.1 


2.2 


3.8 


3.1 


2.6 


21.0 


2.5 


6.0 


7.0 


14 


6.5 


3.3 


4.4 


5.0 


7.8 


2.9 


4.1 


10.4 


5.5 


15 


1.9 


2.3 


3.4 


2.4 


1.3 


3.9 


3.6 


4.4 


5.2 


16 


2.6 


2.2 


2.1 


8.7 


5.1 


2.1 


2.0 


6.0 


2.9 


17 


1.4 


1.8 


13.5 


8.2 


1.6 


3.3 


6.9 


27.0 


23.5 


18 


8.3 


2.3 


7.4 


2.7 


2.0 


3.3 


1.7 


5.9 


14.1 


19 


4.2 


1.8 


6.0 


4.8 


2.5 


2.2 


3.5 


13.2 


24.5 


20 


2.7 


1.4 


4.9 


3.1 


3.6 


3.7 


3.7 


26.4 


14.1 


21 


1.1 


1.2 


9.6 


4.2 


4.4 


5.4 


28.3 


4.4 


13.4 


22 


1.9 


2.8 


3.7 


3.3 


6.3 


3.0 


3.3 


5.8 


10.7 


23 


3.0 


1.9 


5.1 


2.7 


1.7 


2.9 


9.0 


7.2 


18.9 


24 


9.1 


1.0 


3.8 


4.2 


8.8 


2.0 


13.5 


7.3 


6.6 


Total 


72.1 


69.7 


133.3 


96.9 


99.0 


126.6 


133.4 


196.4 


211.6 


Mean 


3.0 


2.9 


5.6 


4.0 


4.1 


5.3 


5.6 


8.2 


8.8 


a 


2.3 


1.8 


3.6 


2.7 


2.1 


4.7 


5.5 


6.4 


6.1 
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Table F.5 Operator II - Tests 2, 3, and 4 





Test 2 


Test 3 


Test 4 


A 


D 


G 


B 


E 


H 


J 


N 


R 


1 


2.0 


1.6 


2.2 


1.2 


2.6 


3.3 


1.2 


2.1 


3.1 


2 


1.2 


1.2 


2.9 


2.2 


2.6 


1.7 


1.2 


1.3 


3.3 


3 


1.4 


2.0 


2.6 


1.4 


5.4 


1.6 


1.0 


1.5 


3.4 


4 


1.3 


5.9 


2.7 


1.4 


3.8 


2.1 


1.5 


2.3 


4.6 


5 


2.5 


3.5 


2.1 


1.6 


2.3 


3.9 


1.4 


1.6 


1.8 


6 


2.0 


2.3 


2.8 


2.2 


1.7 


8.6 


1.2 


2.5 


2.0 


7 


2.1 


2.6 


1.8 


2.2 


2.4 


4.1 


1.1 


2.5 


1.2 


8 


1.4 


1.8 


2.0 


3.4 


2.4 


6.4 


1.0 


1.8 


1.5 


9 


1.6 


2.8 


12.7 


3.4 


2.5 


1.8 


1.1 


1.8 


3.9 


10 


1.9 


2.9 


2.6 


3.6 


1.3 


2.5 


1.0 


1.9 


2.9 


11 


2.2 


3.3 


4.1 


3.0 


1.8 


2.0 


1.0 


1.6 


5.1 


12 


2.6 


2.4 


8.3 


6.0 


1.6 


1.8 


1.3 


1.3 


3.9 


13 


2.0 


13.6 


3.7 


3.6 


1.5 


4.6 


1.8 


1.0 


2.1 


14 


2.6 


3.5 


6.2 


2.8 


8.1 


2.4 


1.5 


1.2 


4.1 


15 


2.0 


2.3 


2.8 


2.4 


3.1 


2.5 


1.4 


1.7 


4.8 


16 


1.8 


2.2 


1.5 


3.0 


3.2 


3.2 


1.8 


4.8 


1.9 


17 


1.3 


2.1 


4.1 


3.4 


3.0 


2.3 


1.7 


1.6 


2.4 


18 


1.5 


3.6 


2.7 


2.8 


3.0 


3.0 


1.7 


2.2 


2.5 


19 


1.0 


2.8 


6.8 


2.0 


4.1 


3.3 


1.8 


1.2 


1.8 


20 


1.0 


2.2 


2.1 


2.3 


2.9 


3.8 


1.8 


1.6 


2.5 


21 


1.1 


1.8 


5.8 


4.1 


4.6 


7.1 


2.0 


1.4 


3.9 


22 


1.8 


2.3 


6.6 


1.9 


4.9 


6.4 


2.5 


1.4 


2.2 


23 


1.8 


1.6 


5.4 


2.0 


2.7 


1.5 


2.8 


0.9 


2.7 


24 


3.7 


2.4 


4.4 


1.7 


3.7 


5.1 


1.8 


1.5 


5.2 


Total 


43.9 


72.6 


98.9 


63.8 


75.3 


85.0 


36.8 


42.7 


72.9 


Mean 


1.8 


3.0 


4.1 


2.7 


3.1 


3.5 


1.5 


1.8 


3.0 


a 


0.6 


2.4 


2.6 


1.0 


1.5 


1.9 


0.5 


0.8 


1.2 
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Table F.6 Operator II - Tests 5, 6, and 7 





Test 5 


Test 6 


Test 7 


K 


0 


P 


C 


F 


I 


L 


M 


Q 


1 


1.1 


2.3 


5.2 


1.3 


1.2 


3.2 


2.0 


2.5 


2.1 


2 


1.0 


2.1 


3.9 


2.7 


1.5 


5.3 


1.7 


2.0 


2.8 


3 


1.2 


2.7 


2.8 


3.0 


2.3 


2.8 


1.7 


2.5 


3.8 


4 


1.3 


2.5 


2.6 


2.1 


1.1 


3.7 


5.4 


2.2 


2.8 


5 


1.1 


2.3 


2.5 


1.8 


2.8 


2.9 


2.9 


2.2 


4.5 


6 


1.2 


2.4 


1.4 


2.8 


2.2 


2.4 


2.0 


6.0 


4.0 


7 


1.0 


2.1 


1.5 


1.7 


1.4 


1.8 


2.3 


2.7 


2.9 


8 


1.1 


2.5 


2.1 


1.7 


1.3 


2.1 


4.6 


2.9 


2.3 


9 


1.7 


1.5 


2.2 


2.6 


1.9 


2.3 


3.4 


2.4 


2.9 


10 


2.0 


1.3 


3.0 


2.4 


1.9 


2.0 


4.0 


2.0 


2.4 


11 


2.0 


1.4 


2.7 


2.0 


1.8 


5.2 


2.0 


2.2 


4.0 


12 


2.3 


4.4 


1.8 


2.0 


2.3 


9.5 


3.3 


2.5 


1.5 


13 


2.5 


2.5 


2.7 


1.6 


1.2 


5.1 


2.2 


2.7 


3.1 


14 


2.2 


1.9 


3.5 


2.3 


1.9 


5.1 


1.4 


2.5 


3.0 


15 


2.4 


1.5 


2.2 


2.7 


1.7 


1.4 


2.6 


2.9 


4.9 


16 


2.4 


2.0 


2.3 


1.6 


1.3 


2.5 


5.3 


3.1 


5.1 


17 


1.9 


2.5 


1.8 


1.6 


2.8 


2.8 


2.5 


3.3 


9.5 


18 


1.6 


3.0 


3.0 


1.6 


2.2 


4.1 


2.5 


3.6 


2.8 


19 


2.0 


3.4 


2.2 


1.4 


1.2 


2.4 


2.4 


2.2 


2.3 


20 


1.5 


1.9 


2.1 


1.6 


1.9 


3.0 


2.4 


2.1 


2.4 


21 


2.1 


1.4 


5.5 


1.8 


2.1 


5.3 


2.7 


1.8 


3.5 


22 


5.1 


1.1 


4.6 


1.7 


1.1 


10.4 


1.7 


2.1 


4.3 


23 


2.4 


1.2 


1.9 


2.0 


1.6 


3.8 


2.0 


2.7 


4.3 


24 


1.9 


1.2 


3.1 


2.1 


3.9 


5.5 


4.1 


1.2 


1.7 


Total 


45.0 


51.3 


66.7 


47.9 


44.6 


94.5 


67.2 


62.3 


82.9 


Mean 


1.9 


2.1 


2.8 


2.0 


1.9 


3.9 


2.8 


2.6 


3.5 


a 


0.8 


0.8 


1.1 


0.5 


0.6 


2.2 


1.1 


0.9 


1.6 
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Table F.7 Operator III - Test 1 





60 ° FOV 


30 ° FOV 


15 ° FOV 


B 


E 


H 


C 


F 


I 


A 


D 


G 


1 


1.0 


3.0 


2.3 


1.0 


5.6 


3.2 


1.4 


1.0 


23.2 


2 


1.2 


8.1 


1.6 


1.1 


4.2 


5.4 


1.1 


1.0 


41.9 


3 


1.1 


4.2 


4.6 


1.3 


5.8 


6.9 


2.0 


19.4 


3.1 


4 


1.0 


3.1 


2.7 


6.5 


4.9 


10.2 


5.5 


16.5 


29.9 


5 


1.1 


4.0 


2.9 


5.4 


6.5 


4.8 


2.4 


3.4 


4.6 


6 


1.1 


3.6 


1.9 


7.1 


5.1 


4.3 


1.8 


3.1 


3.1 


7 


1.0 


3.0 


2.4 


5.5 


3.5 


12.1 


2.9 


4.8 


3.4 


8 


1.0 


5.6 


2.9 


7.8 


4.4 


17.5 


1.3 


1.9 


4.1 


9 


1.1 


1.4 


1.9 


6.1 


4.7 


15.9 


0.9 


2.9 


7.4 


10 


3.9 


1.1 


2.9 


7.2 


7.0 


9.4 


1.1 


2.7 


10.4 


11 


1.0 


1.1 


1.9 


10.7 


4.5 


14.2 


1.4 


10.9 


3.4 


12 


1.7 


1.3 


1.1 


7.2 


5.1 


6.7 


1.7 


3.0 


4.6 


13 


1.4 


11.1 


2.2 


7.7 


8.7 


4.3 


5.4 


2.6 


7.0 


14 


1.4 


3.4 


3.7 


6.3 


10.4 


5.2 


2.5 


2.2 


5.8 


15 


5.5 


1.7 


1.8 


4.8 


4.7 


5.7 


2.9 


2.3 


2.6 


16 


4.5 


1.2 


5.1 


5.7 


4.4 


2.0 


4.3 


3.0 


3.9 


17 


1.6 


1.8 


1.3 


5.5 


6.5 


2.0 


1.7 


3.3 


1.9 


18 


1.3 


4.3 


4.9 


6.7 


6.5 


13.0 


1.1 


17.8 


5.7 


19 


4.9 


1.7 


12.9 


7.3 


4.1 


1.9 


1.0 


3.6 


15.9 


20 


3.0 


3.0 


6.0 


7.4 


4.6 


1.8 


0.9 


3.7 


29.2 


21 


1.3 


3.1 


4.3 


7.5 


5.4 


2.5 


1.0 


2.2 


14.3 


22 


9.1 


2.8 


4.0 


8.9 


3.0 


1.9 


1.3 


6.4 


2.9 


23 


6.3 


2.0 


4.7 


7.1 


6.3 


3.1 


1.5 


0.9 


2.3 


24 


4.0 


3.9 


12.4 


4.7 


3.2 


7.0 


1.4 


1.0 


4.9 


Total 


60.6 


79.6 


92.4 


146.5 


128.9 


160.9 


48.5 


119.6 


235.4 


Mean 


2.5 


3.3 


3.9 


6.1 


5.4 


6.7 


2.0 


5.0 


9.8 


a 


2.1 


2.3 


3.0 


2.3 


1.6 


4.7 


1.3 


5.3 


10.5 
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Table F.8 Operator III - Tests 2, 3, and 4 





Test 2 


Test 3 


Test 4 


K 


N 


Q 


L 


0 


R 


B 


F 


G 


1 


1.4 


2.9 


2.9 


1.0 


1.7 


1.3 


1.9 


1.6 


6.6 


2 


3.1 


1.9 


1.8 


1.0 


1.9 


2.2 


3.7 


1.2 


3.7 


3 


3.4 


3.7 


2.6 


3.3 


2.3 


1.2 


2.7 


1.8 


8.8 


4 


3.4 


1.9 


3.6 


2.8 


3.5 


2.5 


1.2 


4.2 


2.6 


5 


2.0 


1.9 


2.6 


10.5 


1.4 


1.5 


2.1 


4.0 


2.6 


6 


2.9 


2.2 


2.4 


4.5 


4.4 


1.2 


5.8 


2.6 


3.1 


7 


2.0 


1.8 


7.5 


1.2 


1.7 


1.5 


1.2 


1.3 


1.9 


8 


2.0 


3.2 


4.7 


1.8 


2.0 


2.2 


2.2 


1.7 


2.0 


9 


1.4 


2.9 


1.8 


2.2 


7.9 


1.5 


1.7 


2.4 


4.4 


10 


2.1 


1.5 


8.2 


2.4 


6.4 


2.0 


2.8 


1.0 


6.3 


11 


3.9 


1.7 


3.9 


2.1 


2.2 


3.8 


2.2 


1.3 


4.7 


12 


3.1 


2.3 


1.6 


1.6 


1.2 


2.2 


2.2 


1.4 


2.5 


13 


6.9 


1.4 


7.4 


5.9 


3.4 


1.8 


2.5 


1.3 


3.8 


14 


2.7 


2.0 


3.1 


1.0 


1.7 


4.1 


1.4 


1.6 


2.7 


15 


1.6 


2.3 


2.8 


1.0 


1.6 


2.4 


1.5 


1.2 


2.3 


16 


5.3 


3.2 


3.1 


1.1 


4.4 


2.7 


2.1 


1.6 


3.0 


17 


2.5 


2.6 


2.0 


1.2 


2.4 


2.9 


1.7 


2.5 


1.9 


18 


2.0 


2.1 


2.9 


3.4 


6.8 


1.2 


1.6 


2.3 


4.2 


19 


1.9 


2.6 


2.0 


1.6 


3.1 


2.3 


2.7 


1.6 


3.7 


20 


2.1 


2.4 


2.8 


10.4 


1.2 


5.7 


2.3 


1.3 


2.1 


21 


2.3 


1.5 


3.9 


2.2 


3.0 


3.3 


2.5 


1.7 


2.1 


22 


2.9 


1.1 


3.1 


1.7 


2.0 


2.2 


1.9 


1.7 


2.5 


23 


2.6 


1.8 


6.3 


1.0 


1.1 


1.8 


2.6 


1.1 


1.4 


24 


2.2 


1.2 


2.2 


1.0 


1.9 


5.9 


2.2 


1.3 


5.1 


Total 


65.7 


52.3 


85.1 


65.6 


69.3 


59.6 


54.8 


43.9 


84.1 


Mean 


2.7 


2.2 


3.5 


2.7 


2.9 


2.5 


2.3 


1.8 


3.5 


a 


1.2 


0.6 


1.9 


2.6 


1.8 


1.3 


0.9 


0.8 


1.7 
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Table F.9 Operator III - Tests 5, 6, and 7 





Test 5 


Test 6 


Test 7 


C 


D 


H 


J 


M 


P 


A 


E 


I 


1 


1.0 


2.1 


1.7 


1.1 


2.6 


1.1 


1.6 


2.5 


2.1 


2 


1.0 


1.0 


1.0 


0.9 


1.5 


3.6 


1.9 


2.4 


6.9 


3 


1.0 


2.1 


2.2 


4.3 


2.3 


2.1 


2.1 


5.5 


6.3 


4 


1.3 


6.3 


2.3 


1.6 


4.0 


2.4 


2.0 


2.5 


4.0 


5 


2.2 


1.5 


2.1 


1.2 


7.2 


1.5 


1.2 


3.5 


4.7 


6 


4.7 


1.5 


2.1 


1.1 


2.5 


4.8 


2.1 


2.9 


2.2 


7 


1.7 


2.2 


1.6 


1.0 


1.9 


2.0 


2.4 


6.3 


4.4 


8 


4.5 


1.7 


3.3 


1.0 


1.7 


4.2 


1.5 


4.6 


3.9 


9 


4.2 


2.3 


1.7 


1.1 


1.4 


2.9 


2.4 


2.6 


2.7 


10 


5.8 


1.2 


3.9 


1.0 


6.0 


4.1 


2.6 


1.9 


2.7 


11 


2.0 


2.4 


3.1 


1.4 


2.6 


4.2 


2.1 


2.0 


7.6 


12 


5.0 


2.3 


1.8 


1.0 


3.8 


3.9 


2.8 


3.1 


6.9 


13 


2.4 


2.4 


2.5 


0.9 


3.4 


4.7 


5.7 


1.9 


4.7 


14 


2.0 


2.4 


1.5 


1.2 


3.3 


6.2 


2.3 


1.8 


1.6 


15 


1.7 


2.6 


1.9 


1.0 


2.9 


3.9 


1.9 


1.7 


1.8 


16 


1.5 


2.0 


2.2 


1.9 


2.6 


2.7 


3.1 


2.0 


3.6 


17 


1.4 


1.9 


2.7 


1.6 


2.2 


3.0 


3.6 


1.3 


3.4 


18 


1.5 


2.4 


2.4 


1.1 


3.4 


2.7 


1.1 


3.2 


7.3 


19 


2.5 


2.2 


2.6 


5.7 


3.6 


3.4 


2.4 


2.1 


3.9 


20 


1.1 


4.1 


2.3 


1.5 


3.8 


3.8 


2.6 


3.8 


2.1 


21 


1.2 


1.1 


4.6 


1.4 


2.6 


4.3 


1.7 


10.6 


4.1 


22 


1.1 


1.0 


5.1 


1.6 


2.4 


7.1 


2.6 


3.6 


1.9 


23 


2.6 


1.0 


2.2 


0.9 


1.9 


2.6 


5.2 


2.9 


2.2 


24 


2.5 


1.0 


1.9 


1.7 


1.7 


1.6 


2.5 


2.9 


2.8 


Total 


56.0 


50.7 


58.6 


37.5 


71.1 


82.8 


59.6 


77.6 


93.7 


Mean 


2.3 


2.1 


2.4 


1.6 


3.0 


3.5 


2.5 


3.2 


3.9 


a 


1.4 


1.1 


0.9 


1.1 


1.3 


1.4 


1.0 


1.9 


1.8 
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Table F . 10 Operator IV - Test 1 





60 ° FOV 


30 ° FOV 


15 ° FOV 


K 


N 


Q 


L 


O 


R 


J 


M 


P 


1 


3.8 


2.4 


2.8 


4.0 


5.3 


8.2 


7.6 


4.7 


3.7 


2 


3.8 


4.8 


3.2 


4.4 


4.8 


8.5 


8.6 


3.1 


5.4 


3 


3.1 


2.9 


4.8 


3.8 


7.6 


5.4 


6.7 


13.8 


5.4 


4 


5.0 


2.7 


2.7 


6.3 


4.3 


7.0 


4.8 


6.6 


11.4 


5 


5.5 


4.1 


3.4 


7.2 


4.1 


8.5 


6.4 


30.9 


5.4 


6 


3.9 


5.2 


2.1 


4.0 


2.6 


2.1 


5.3 


5.2 


29.6 


7 


6.8 


2.7 


2.4 


4.4 


2.7 


6.5 


4.3 


7.6 


5.9 


8 


5.2 


5.0 


3.6 


8.2 


1.8 


4.9 


3.0 


2.5 


36.6 


9 


2.9 


4.3 


2.5 


2.7 


1.5 


19.5 


2.1 


5.6 


8.4 


10 


2.7 


3.6 


2.4 


3.9 


1.4 


5.2 


4.9 


4.5 


17.7 


11 


2.4 


4.9 


2.2 


4.3 


1.1 


17.8 


4.6 


3.2 


11.0 


12 


2.5 


2.6 


2.2 


2.6 


3.0 


18.9 


3.1 


5.5 


8.4 


13 


3.4 


1.5 


3.9 


2.6 


3.8 


12.1 


10.5 


22.0 


12.4 


14 


3.6 


3.2 


3.4 


4.1 


4.4 


16.4 


13.9 


23.6 


21.2 


15 


3.7 


1.9 


3.2 


2.2 


3.8 


7.2 


4.4 


5.0 


5.5 


16 


2.5 


2.2 


3.7 


1.7 


1.4 


8.8 


10.2 


5.0 


6.9 


17 


2.8 


2.4 


2.4 


2.0 


5.1 


6.4 


4.1 


4.9 


5.0 


18 


4.2 


1.8 


4.2 


3.4 


5.1 


6.6 


10.0 


19.8 


4.9 


19 


3.1 


2.2 


1.7 


3.0 


1.5 


2.2 


12.8 


6.5 


7.2 


20 


4.7 


2.3 


8.6 


2.8 


2.5 


7.3 


12.1 


7.4 


4.7 


21 


5.3 


3.6 


2.7 


2.1 


17.3 


9.3 


4.6 


1.4 


5.0 


22 


2.8 


4.7 


3.9 


2.0 


5.4 


7.3 


3.2 


4.1 


6.9 


23 


2.3 


3.2 


6.7 


4.0 


3.5 


11.8 


5.9 


4.2 


12.2 


24 


2.7 


1.4 


4.9 


5.1 


4.0 


2.2 


6.3 


3.0 


8.8 


Total 


88.8 


75.6 


83.4 


90.6 


98.0 


210.3 


159.7 


200.1 


249.6 


Mean 


3.7 


3.2 


3.5 


3.8 


4.1 


8.8 


6.7 


8.3 


10.4 


a 


1.2 


1.2 


1.5 


1.6 


3.2 


4.9 


3.3 


7.6 


8.1 
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Table F. 1 1 Operator IV - Tests 2, 3, and 4 





Test 2 


Test 3 


Test 4 


B 


E 


H 


C 


F 


I 


K 


0 


P 


1 


1.1 


1.7 


2.1 


1.1 


1.6 


2.2 


1.2 


1.1 


3.0 


2 


2.0 


2.3 


1.3 


1.2 


4.0 


2.6 


2.3 


1.8 


4.3 


3 


1.6 


2.3 


3.7 


1.0 


2.3 


1.9 


1.8 


2.6 


2.0 


4 


1.7 


2.9 


2.9 


1.2 


1.8 


6.1 


3.6 


2.8 


2.4 


5 


1.7 


3.7 


4.9 


1.8 


1.3 


3.1 


2.1 


2.3 


1.6 


6 


1.4 


2.2 


2.2 


1.5 


1.7 


2.4 


1.6 


1.9 


1.6 


7 


1.3 


1.8 


2.9 


1.3 


1.9 


1.8 


1.7 


1.3 


1.9 


8 


2.3 


2.3 


2.1 


2.3 


1.9 


1.4 


1.1 


3.0 


2.5 


9 


2.1 


2.0 


2.0 


1.9 


1.8 


2.3 


1.2 


1.8 


2.7 


10 


3.8 


2.2 


1.7 


5.9 


1.6 


2.1 


1.4 


1.0 


3.2 


11 


3.9 


2.6 


1.7 


2.0 


1.8 


7.6 


1.6 


4.3 


1.3 


12 


2.0 


1.4 


6.3 


1.8 


3.2 


5.4 


1.8 


1.6 


8.4 


13 


1.4 


1.0 


2.3 


1.4 


1.6 


5.5 


1.4 


1.4 


3.3 


14 


1.4 


1.6 


2.5 


1.1 


2.2 


6.3 


1.2 


1.8 


1.8 


15 


1.2 


1.2 


2.9 


1.3 


3.6 


1.8 


1.5 


1.8 


2.2 


16 


1.3 


1.7 


4.0 


1.4 


2.7 


2.2 


1.8 


1.3 


5.6 


17 


5.0 


1.2 


3.1 


1.2 


1.8 


1.8 


1.8 


1.8 


4.0 


18 


1.5 


2.0 


3.7 


1.4 


1.2 


2.5 


1.5 


1.4 


4.7 


19 


4.1 


1.9 


6.7 


1.3 


1.4 


1.2 


1.7 


1.1 


4.1 


20 


2.1 


1.9 


2.1 


1.6 


1.1 


2.0 


1.6 


1.3 


2.3 


21 


2.0 


2.0 


4.0 


1.1 


1.1 


2.4 


4.7 


1.4 


1.8 


22 


4.3 


1.7 


3.1 


1.3 


1.8 


1.9 


1.9 


1.4 


4.1 


23 


1.4 


1.1 


2.2 


1.1 


1.5 


1.5 


3.4 


1.0 


1.1 


24 


1.0 


1.1 


1.8 


1.0 


1.0 


1.8 


2.6 


1.0 


3.1 


Total 


51.9 


45.9 


72.3 


38.1 


45.9 


69.9 


46.5 


42.4 


73.0 


Mean 


2.2 


1.9 


3.0 


1.6 


1.9 


2.9 


1.9 


1.8 


3.0 


a 


1.1 


0.6 


1.4 


1.0 


0.8 


1.8 


0.8 


0.8 


1.6 
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Table F. 12 Operator IV - Tests 5, 6, and 7 





Test 5 


Test 6 


Test 7 


L 


M 


Q 


A 


D 


G 


J 


N 


R 


1 


1.1 


1.7 


1.9 


1.9 


1.5 


2.6 


1.2 


3.3 


1.2 


2 


1.2 


1.1 


1.1 


2.9 


1.7 


2.4 


1.4 


3.5 


3.9 


3 


1.0 


1.8 


1.9 


2.0 


1.4 


2.1 


2.3 


3.0 


7.5 


4 


1.4 


1.4 


1.7 


1.3 


6.5 


2.9 


1.7 


2.6 


2.8 


5 


1.3 


1.1 


1.9 


1.2 


3.1 


1.6 


1.3 


2.3 


1.8 


6 


1.2 


1.1 


1.7 


1.3 


1.2 


1.6 


1.3 


2.4 


4.4 


7 


1.9 


1.4 


6.1 


1.6 


1.4 


1.9 


1.4 


2.6 


2.7 


8 


1.3 


1.1 


1.7 


1.3 


1.4 


2.8 


1.4 


2.6 


3.0 


9 


1.2 


1.3 


1.6 


2.2 


2.1 


2.7 


1.5 


1.4 


3.0 


10 


1.8 


1.5 


4.1 


2.0 


2.5 


4.8 


1.2 


2.0 


2.0 


11 


1.4 


1.9 


1.5 


1.0 


1.9 


2.2 


2.1 


1.6 


4.2 


12 


1.4 


1.8 


1.6 


1.1 


2.0 


3.4 


2.6 


2.0 


4.3 


13 


3.9 


1.4 


1.9 


1.0 


2.7 


4.0 


2.8 


1.2 


3.8 


14 


1.1 


2.4 


2.1 


1.2 


2.4 


8.1 


2.6 


2.0 


10.0 


15 


1.0 


2.0 


11.4 


1.1 


1.4 


3.1 


2.0 


1.4 


2.6 


16 


1.0 


1.6 


2.1 


1.0 


6.2 


2.2 


3.7 


2.4 


2.7 


17 


5.2 


1.5 


3.9 


1.1 


2.6 


4.8 


2.5 


1.7 


2.1 


18 


1.4 


3.0 


5.1 


0.9 


2.4 


4.0 


1.3 


1.8 


2.4 


19 


1.3 


1.6 


2.3 


1.1 


2.9 


5.0 


1.3 


2.0 


1.2 


20 


1.3 


1.7 


3.1 


2.0 


2.2 


1.7 


2.5 


1.8 


2.8 


21 


1.5 


0.9 


3.3 


1.7 


2.1 


1.8 


3.2 


2.3 


3.1 


22 


1.3 


1.0 


2.5 


1.3 


1.7 


2.4 


1.8 


1.3 


2.1 


23 


1.1 


1.0 


1.4 


1.0 


1.2 


1.5 


1.5 


2.0 


2.8 


24 


1.0 


1.0 


2.4 


5.0 


1.1 


3.1 


2.7 


1.4 


1.9 


Total 


37.5 


36.5 


68.3 


38.5 


55.7 


72.9 


47.4 


50.7 


78.2 


Mean 


1.6 


1.5 


2.8 


1.6 


2.3 


3.0 


2.0 


2.1 


3.3 


o 


1.0 


0.5 


2.2 


0.9 


1.3 


1.5 


0.7 


0.6 


1.9 
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APPENDIX G 



Operator Learning Curves 



Estimation of the operators' learning curves involved four assumptions. 

• The learning curves had positive slope at all times. 

• The learning curves were smooth functions. 

• The learning curves were exponential functions. 

• For a specified field of view, operator learning was reflected primarily in the 

standard deviation of the normalized course times (a smoothness measure). 

Each operator drove 21 courses with a 60° field of view, three for each of the seven 
tests. The procedure below was applied to the normalized data for each operator to estimate 
his 60° FOV learning curve. 

• The standard deviations were entered into a 3 x 7 matrix. Each row represented 

a specific window width |3 as discussed in section 5.2 (row 1 = 15°, row 2 
= 30°, and row 3 = 60°). Each column corresponded to a given test (column 
1 = test 1, column 2 = test 2, etc.). 

• The standard deviations for the three window widths for each test were summed 

and entered into a row vector of seven elements. Each column continued to 
represent a given test as above. This step yielded the summed standard 
deviations (SSD). 

• The row vector elements were plotted with summed standard deviations on the 

vertical axis versus test number on the horizontal axis. 

• An exponential curve using the method of least squares was fitted to the plot by 

linearizing the exponential form through taking logarithms [G-l]. This 
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defined the operator's 60° FOV learning curve. 



Figures G.l through G.4 show the learning curves (graphed as lines) versus the 
test number. 
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8 




Test -> 1 2 3 4 5 6 7 

Figure G. 1 Operator 1 60° FOV learning curve 




Figure G.2 Operator II 60° FOV learning curve 
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Test -> 1 2 3 4 5 6 7 

Figure G.3 Operator m 60° FOV learning curve 
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Test -> 1 2 3 4 5 6 7 

Figure G.4 Operator IV 60° FOV learning curve 
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